Case: 22-10312, 04/25/2023, ID: 12702996, DktEntry: 36-31, Page 1 of 95 


EXHIBIT 30 


Case: 2281-68. 0226642020 20th mei GHG, Pitéenhtty/AH2I, Page 2 of 93 


Exhibit 27 


Case:22816602 266420) 20 230d meiGHs, Pikenhtty/AG2D , Page 2 of 94 


thera Meo 1701 Page Mill Road P 650.838.9292 theranos.com 


Palo Alto, CA 94304 F 650.838.9165 


April 1, 2016 
BY MESSENGER 


Ms. Karen Fuller 

State Oversight and CLIA Branch 

Division of Survey and Certification 
Centers for Medicare and Medicaid Services 
90 Seventh Street, Suite 5-300 (SW) 

San Francisco, CA 94103-6707 


Re: | Updated Response of Theranos, Inc. to the March 18, 2016 Letter From Karen 
Fuller To Dr. Sunil Dhawan, Elizabeth Holmes, and Sunny Balwani (CLIA Number 
05D2025714) 


Dear Ms. Fuller, 


This letter provides Theranos, Inc.’s response to your letter of March 18, 2016. We respond to 
each of the reasons why CMS found that our response to a specific deficiency was insufficient by 
providing additional information and supporting documentation, which we believe demonstrate 
that our Newark, California laboratory has come into Condition-level compliance and abated the 
immediate jeopardy. 


Based on the productive discussions we have had with Sarah Bennett and Gary Yamamoto about 
your March 18 letter, this response aims to ensure that there are not any miscommunications 
about important issues that we have learned were unclear to CMS from our initial submission. In 
addition, this response reflects work that the laboratory, under the direction of its new leadership, 
has done to address CMS’s feedback, including, for example, performing thorough re-review of 
data and further revising procedures. 


As set forth in this detailed response and through the enclosed supporting documentation: 


1. The laboratory has put in place new leadership, including a new full-time lab director and 
a new full-time clinical consultant, who is a co-director under California’s regulations. 
The new laboratory director, Dr. Kingshuk Das, has been on-site full-time as of March 
14, 2016. He has extensive clinical laboratory experience, including as an Associate 
Medical Director of UCLA’s Clinical Laboratories between August 2013 and March 
2016. The new clinical consultant, Dr. Don Tschirhart, has been on-site full-time as of 
March 14, 2016. He has over 15 years of experience as a hospital lab director. 


2. The laboratory’s new leadership firmly believes that the laboratory will be able to ensure 
the deficient practices will not recur through its robust new quality systems and through 
the intense oversight being provided by the laboratory’s new quality monitoring and 
improvement program and new audit procedures. Indeed, the laboratory’s new audit 
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procedures require both monthly tracer audits of pre-analytic, analytic and post-analytic 
processes and quarterly audits of the laboratory’s quality systems. The results of these 
audits are then scrutinized at the laboratory’s monthly QMPI Program meetings. 


aS) 


Many of the issues raised in your March 18 letter appear to be premised on CMS’s belief 
that the laboratory is running a number of tests that it actually stopped running either 
before or during the survey, and has not run since. We have clarified this issue below. 


4. The laboratory has undertaken aggressive corrective actions. For example, out of an 
extreme abundance of caution and based on its dissatisfaction with prior QA oversight, 
the laboratory voided all results reported for the assays run on the Theranos Proprietary 
System 3.5 (TPS) in 2014 and 2015 and all reported PT/INR tests run on the Siemens 
sees BCS XP instrument that went into use in October 2014 through September of 
2015. 


We believe that these considerable actions, when viewed together with the other information and 
documentation provided herein and in our initial submission, should leave no doubt that the 
Newark laboratory has come into Condition-level compliance and abated immediate jeopardy. 
For that reason, we submit that CMS should approve the laboratory’s submission, and thereafter 
perform a revisit at which CMS will see that the laboratory’s new leadership has implemented, 
enforced, and monitored the corrective actions. 


We welcome further engagement with CMS on any of the information we have submitted to 
date, including the opportunity to address any questions or concerns CMS may have in real-time, 
in order to avoid any unnecessary delay or work for CMS and pursuant to our efforts to respond 
comprehensively to all issues raised by CMS. 


Please note that these materials contain confidential commercial information and trade secrets 
that are statutorily exempt from disclosure under the Freedom of Information Act, 5 U.S.C. § 
552(b)(4). We are providing this highly sensitive information to CMS in order to fully respond 
to the agency’s inquiries. However, this is information that Theranos closely protects from 
public disclosure, and any disclosure by the agency of this information is likely to cause the 
company irreparable harm. As a result, in providing this information, we respectfully request the 
opportunity to provide redactions should you decide to disclose any of the documents to any 
member of the public for any reason, under FOIA procedures or otherwise. In addition, if the 
agency incorporates the substance of any of this information into correspondence or other 
agency-generated documents that may be publicly disclosed, we request the opportunity to redact 
the information from those documents as well. We also ask that you take steps to protect these 
documents from unauthorized disclosure 


' A list of all accessions for which corrected reports have been issued is enclosed herein in 
Exhibit MM. 
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D2094 


D2094 — March 18 Letter Statement 1: “Some documents pertaining to this deficiency 
referenced in the submission were not included. Specifically, the submission references ‘Ex. A, 
Tab 4, § 7.1.2.2.d.2.’ We found no such reference contained in the materials provided to CMS. 
In addition, Exhibit (Ex.) O, Tab 2 states: ‘The QC data are presented here: <Exhibit D Tab 7, 
Tab 8>.’ We found no ‘Tab 7’ or ‘Tab 8’ in Ex. D.” 


Theranos Response to Statement 1 in D2094: The citation reference to Ex. A, Tab 4, § 
7.1.2.2.d.2 was an inadvertent transcription. The citation reference Ex. D, Tabs 7 and 8 were 
also inadvertent transcription errors, but the QC data were included in the initial submission 
in Exhibit O. We have enclosed another copy of that QC data for ALP with this letter at Ex. 
JJ, Tab 3B. 


D2094 — March 18 Letter Statement 2: “A/though the laboratory's submitted protocol 
indicates that ungraded proficiency testing (PT) results will be evaluated, the submitted protocol 
does not explain how an investigation is performed and who must sign an investigation of 
ungraded PT samples.” 


Theranos Response to Statement 2 in D2094: The lab has revised its proficiency testing 
procedure (i) to indicate the protocol used to evaluate ungraded proficiency testing (PT); (ii) 
to explain how an investigation of ungraded PT is performed; and (iii) to explain who must 
sign an investigation of ungraded PT samples. (Ex. BB, Tab 3). The revised procedure 
requires the laboratory to use its Proficiency Testing Investigation and Corrective Action 
form to investigate ungraded PT, which must be approved by the lab director. (Ex. BB, Tab 
3§ 9.4, F-1). Training on this revised SOP has occurred. (Ex. BB, Tabs 3B-3C). 


D2094 — March 18 Letter Statement 3: “Jn the submission, the laboratory concludes: 


- ‘Not enough patient data available for meaningful analysis’ 
- ‘No evidence of systemic errors’ 
- ‘No patient impact expected’ 


However, no information as to how the laboratory came to these conclusions related to patient 
outcomes was submitted. Documentation contained in in Ex. O, Tab 2 also compares the ‘Range 
of Means’ with no explanation as to what this refers to or how it correlates to the laboratory's 
ungraded results. The submission merely indicates that ‘the lab has investigated this ungraded 
PT event for ALP [alkaline phosphatase] and has documented its investigation and conclusion.” 


Theranos Response to Statement 3 in D2094: Pursuant to its revised proficiency testing 
procedure, the lab has documented its investigation of the ungraded ALP event for the 3rd 
event of 2014 using the Proficiency Testing Investigation and Corrective Action form. (Ex. 
JJ, Tab 3A). That documentation explains how the laboratory came to its conclusions related 
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to patient outcomes. (Ex. JJ, Tab 3A). Additionally, the laboratory has revised and clarified 
its analysis, including by removing the phrase “Range of Means.” (Ex. AA, Tab 13). 


D2094 — March 18 Letter Statement 4: “Furthermore, a Quality Monitoring and Processing 
Improvement (OMPI), Program Meeting Agenda, CL FRM-00045-F'1, that was submitted as part 
of Ex. A, Tab 13 and only includes PT for Alternative Proficiency Assessment (APA). Based on 
information included in the submitted agenda, all PT issues were not addressed as required in 
CL QOP-00045, Revision A (Ex. A, Tab 12) in Section 7.2.2.6 and 7.2.2.7.” 


Theranos Response to Statement 4 in D2094: The lab has revised the QMPI Program 
procedures and meeting agenda to make clear that the agenda includes all PT issues as 
required in CL QOP-00045. (Ex. BB, Tab 7 § 7.5.2).? 


D2094 — March 18 Letter Statement 5: “Zo ensure the deficient practice does not recur, the 
laboratory indicated that quarterly audits will be performed and suggested that the audits results 
would be reviewed within the laboratory’s OMPI Program. However, the laboratory did not 
establish the procedure by which these quarterly audits are to be conducted. In the submission, 
the laboratory indicates that a ‘tracer audit may [emphasis added] be used, but did not provided 
[sic] a protocol for a ‘tracer audit,’ the means by which a ‘tracer audit’ would be documented, 
and whether the results of a ‘tracer audit’ would be the information reviewed by the OMPI 
Program.” 


Theranos Response to Statement 5 in D2094: The laboratory has revised its audit 
procedures to establish the procedures by which audits are to be conducted. (Ex. BB, Tab 
8 § 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-8C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a 
tracer audit; and require that tracer audits be documented using the tracer audit form, CL 
FRM 00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require 
a quarterly audit to cover the quality management systems; provide the protocol for 
quarterly audits; and require that quarterly audits are documented using the audit record 
form, CL QOP 00046 F1. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been 
revised to make clear that “[b]oth tracer and quarterly audits are reviewed, analyzed and 
presented based on findings and analysis” during the QMPI Program meetings. (Ex. BB, 
Tab 7§ 7.5).> Training on the revised QMPI procedures has occurred. (Ex. BB, Tabs 
7B-7C). 


Note, the prior version of the QMPI QOP that was included with the submission stated that 
QMPI Meetings would include a review of the “schedule” and “results” for “Proficiency Tests”. 
(Ex. A, Tab 12, § 7.2.2.5). 


3 Note, the prior version of the QMPI Meeting Agenda that was included with the submission 
required a review of audit “Findings.” (Ex. A, Tab 12, § 9.) 
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D2128 


D2128 — March 18 Letter Statement 1: “Zhe submission references ‘Ex. A, Tab 4, § 
7.1.2.2.d.2.’ We found no such reference contained in the materials provided to CMS.” 


Theranos Response to Statement 1 in D2128: The citation reference to Ex. A, Tab 4, § 
7.1.2.2.d.2 was an inadvertent transcription error and should not have been included. 


D2128 — March 18 Letter Statement 2 “A/though the laboratory’s submitted protocol 
indicates that unsatisfactory PT results would be investigated using CL FRM-00006-F 1, the 
laboratory provided no documentation showing that the required form was used to investigate 
the unsatisfactory PT result. The laboratory also states that the ‘lab has investigated the one 
unacceptable challenge and documented its investigation and conclusions’; however, CL FRM- 
00006-F'1 documenting the investigation was not submitted per the laboratory’s protocol.” 


Theranos Response to Statement 2 in D2128: We note that there was not an 
“unsatisfactory PT result” reported by the American Proficiency Institute for the “Blood Cell 
ID (Educational) 2nd event of 2014, because this educational event was “Not Graded.” (Ex. 
JJ, Tab 2). Pursuant to its revised proficiency testing procedure, the laboratory has 
documented its investigation of this “Blood Cell ID (Educational)” 2nd event of 2014 that 
was “Not Graded” using the Proficiency Testing Investigation and Corrective Action form. 
(Ex. JJ, Tab 2). 


Likewise, pursuant to its revised proficiency testing procedure, the laboratory has 
documented its investigation of the one unacceptable challenge for the “Blood Cell 
Identification” 2nd event of 2014 using the Proficiency Testing Investigation and Corrective 
Action form. (Ex. JJ, Tab 2A). 


D2128 — March 18 Letter Statement 3: “Jn the submission, the laboratory concludes: 


- ‘Not enough patient data available for meaningful analysis’ 
- ‘No evidence of systemic errors’ 
- ‘No patient impact expected’ 


However, no information as to how the laboratory came to these conclusions related to patient 
outcomes was submitted. Documentation contained in Ex O, Tab 3 shows that the laboratory 
received a passing score of 80% for blood cell identification.” While this is true, the CLIA 
regulations require that all unacceptable results have remedial action taken and that such 
remedial action be documented. The submission does not contain evidence of remedial action.” 


Theranos Response to Statement 3 in D2128: Pursuant to its revised proficiency testing 
procedure, the laboratory has documented its investigation of the “Blood Cell ID 
(Educational)” 2nd event of 2014 that was “Not Graded” using the Proficiency Testing 
Investigation and Corrective Action form. (Ex. JJ, Tab 2). That documentation explains how 
the laboratory came to its conclusions related to patient outcomes. (Ex. JJ, Tab 2). 
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Pursuant to its revised proficiency testing procedure, the laboratory has documented its 
investigation of the one unacceptable challenge for “Blood Cell Identification” 2nd event of 
2014 using the Proficiency Testing Investigation and Corrective Action form. (Ex. JJ, Tab 
2A). That documentation explains how the laboratory came to its conclusions related to 
patient outcomes. (Ex. JJ, Tab 2A). 


D2128 — March 18 Letter Statement 4: “Zo ensure the deficient practice does not recur, the 
laboratory indicated that quarterly audits will be performed and suggested that the audits results 
would be reviewed within the laboratory’s OMPI Program. However, the laboratory did not 
establish the procedure by which these quarterly audits are to be conducted. In the submission, 
the laboratory indicates that a ‘tracer audit may [emphasis added [in the letter]] be used, but 
did not provided [sic] a protocol for a ‘tracer audit,’ the means by which a ‘tracer audit’ would 
be documented, and whether the results of a ‘tracer audit’ would be the information reviewed by 
the OMPI Program.” 


Theranos Response to Statement 4 in D2128: The laboratory has revised its audit 
procedures to establish the procedures by which audits are to be conducted. (Ex. BB, Tab 8 § 
8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-8C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 F1. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]oth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB, Tab 7 § 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tab 7B-7C). 


DS5024: 


The March 18 Letter states: “See our reviews of D5403, D5437, D5447, D5469, D5481, D5779, 
and D5801.”" Accordingly, we refer you to our responses to those reviews. 


D5217 


D5217 — March 18 Letter Statement 1: “Some documents pertaining to this deficiency 
referenced in the submission were not included. Specifically, the submission references ‘Ex. A, 
Tab 4, § 7.1.2.2.d.2.’ We found no such reference contained in the materials provided to CMS. 
In addition, Exhibit (Ex.) O, Tab 2 states: ‘The OC data are presented here: <Exhibit D Tab 7, 
Tab 8>.’ We found no ‘Tab 7’ or ‘Tab 8’ in Ex. D.” 


Theranos Response to Statement 1 — D5217: The citation reference to Ex. A, Tab 4, § 
7.1.2.2.d.2 was an inadvertent transcription error and should not have been included. The 
citation reference Ex. D, Tabs 7 and 8 were also inadvertent transcription errors, but the QC 
data were included in the submission in Exhibit O. We have enclosed another copy of the 
QC data for Troponin with this letter at Ex. JJ, Tab 1B. 
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D5217 — March 18 Letter Statement 2: “Jn the submission, the laboratory concludes: 


- ‘Not enough patient data available for meaningful analysis’ 
- ‘No evidence of systemic errors’ 
- ‘No patient impact expected’ 


However, no information as to how the laboratory came to these conclusions related to patient 
outcomes was submitted. 


Theranos Response to Statement 2 — D5217: The lab has revised its conclusions for the 
ungraded Troponin 2nd event of 2014; those conclusions are consistent with CMS’s review 
in response to D5217 Statement 3 below. Pursuant to its revised proficiency testing 
procedure, the laboratory has documented its investigation and conclusion of the ungraded 
Troponin 2nd event of 2014 using the Proficiency Testing Investigation and Corrective 
Action form. (Ex. JJ, Tab 1A). That documentation explains how the laboratory came to its 
conclusions related to patient outcomes. (Ex. JJ, Tabs 1A-1B). 


D5217 — March 18 Letter Statement 3: “Zhe laboratory states in Ex. O, Tab I that no peer 
data was available for the Siemens Immulite 2000. Based on lack of a peer group, the 
laboratory should compare its results to ‘All Participants’ values for troponin. Review of the 
laboratory's values versus the ‘All Participants’ values shows that four of five of the 
laboratory’s troponin values for the second proficiency testing event of 2014 were unacceptable 
when compared to the ‘All Participants’ acceptable ranges. It appears the laboratory reviewed 
all of the mean values for all peer, instrument and method groups in order to determine an 
inappropriate acceptable range for its samples based on data not related to the ‘All 
Participants’ values.” 


Theranos Response to Statement 3 — D5217: Pursuant to its revised proficiency testing 
procedure, the laboratory has documented its investigation and conclusion of the ungraded 
Troponin 2nd event of 2014 using the Proficiency Testing Investigation and Corrective 
Action form. (Ex. JJ, Tab 1A). That documentation reflects the laboratory’s revised 
investigation, analysis, and conclusions. (Ex. JJ, Tabs |A-1B). Based on its investigation, 
the laboratory has concluded that its values versus the “All Participants” values show that 
four of five of the laboratory’s troponin values for the second proficiency testing event of 
2014 did not constitute passing values for this PT event. (Ex. JJ, Tabs 1A-1B). The enclosed 
documentation describes the investigation and remedial action taken to address the 
unacceptable values. (Ex. JJ, Tabs 1A-1B). As explained in that documentation, the 
laboratory has, out of an abundance of caution, issued corrected reports voiding all affected 
Troponin results. (Ex. JJ, Tab 1A). Many corrected reports have been transmitted, and the 
remainder are being transmitted. (Ex. KK). Transmission will be complete by March 31, 
2016. The remainder of the transmittals and confirmations of receipt will be provided to 
CMS under separate cover. 


D5217 — March 18 Letter Statement 4: “Zo ensure the deficient practice does not recur, the 
laboratory indicated that quarterly audits will be performed and suggested that the audits results 
would be reviewed within the laboratory’s OMPI Program. However, the laboratory did not 
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establish the procedure by which these quarterly audits are to be conducted. In the submission, 
the laboratory indicates that a ‘tracer audit may [emphasis added [in the letter]] be used, but 
did not provided [sic] a protocol for a ‘tracer audit,’ the means by which a ‘tracer audit’ would 
be documented, and whether the results of a ‘tracer audit’ would be the information reviewed by 
the OMPI Program.” 


Theranos Response to Statement 4 — D5217: The laboratory has revised its audit 
procedures to establish the procedures by which audits are to be conducted. (Ex. BB, Tab 8). 
Training on these procedures has occurred. (Ex. BB, Tabs 8B-C). The revised procedures 
establish that tracer audits must occur monthly; provide the protocol for a tracer audit; and 
require that tracer audits be documented using the tracer audit form, CL FRM 00046 F2. 

(Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a quarterly audit to 
cover the quality management systems; provide the protocol for quarterly audits; and require 
that quarterly audits are documented using the audit record form, CL QOP 00046 F1. (Ex. 
BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear that “[b]oth 
tracer and quarterly audits are reviewed, analyzed and presented based on findings and 
analysis” during the QMPI Program meetings. (Ex. BB Tab 7§ 7.5). Training on the revised 
QMPI procedures has occurred. (Ex. BB, Tab7B-C). 


DS311 


D5311 — March 18 Letter Statement 1: “Zhe submission references ‘lx. A, Tab 4, 7.1.2.2.d.2.’ 
We found no such reference contained in the materials provided to CMS.” 


Theranos Response to Statement 1 — D5311: The citation reference to Ex. A, Tab 4, § 
7.1.2.2.d.2 was an inadvertent transcription error and should not have been included. 


D5311 — March 18 Letter Statement 2: “Zhe laboratory concluded that ‘this issue is not likely 
to affect patients’ because its mislabel error rate is comparable to published industry mislabel 
error rates. No other information related to possible patient outcomes was provided. Even 
though the laboratory’s error rate may be comparable, the laboratory must still pursue any 
corrective action(s) for patients that may have been affected by this deficient practice.” 


Theranos Response to Statement 2 — D5311: The laboratory has revised its patient impact 
assessment analysis and conclusions to address CMS’s review. (Ex. AA, Tab 1). Among 
other things, the revised assessment shows that laboratory took corrective action in 2014 and 
2015 for patients that may have been affected by offering a redraw and retest at no charge. 
(Ex. AA, Tab 1; Ex. HH, Tab 10). 


D5311 — March 18 Letter Statement 3: “Zo ensure the deficient practice does not recur, the 
laboratory indicated that quarterly audits will be performed and suggested that the audits results 
would be reviewed within the laboratory’s OMPI Program. However, the laboratory did not 
establish the procedure by which these quarterly audits are to be conducted. In the submission, 
the laboratory indicates that a ‘tracer audit may [emphasis added [in the letter]] be used, but 
did not provided [sic] a protocol for a ‘tracer audit,’ the means by which a ‘tracer audit’ would 
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be documented, and whether the results of a ‘tracer audit’ would be the information reviewed by 
the OMPI Program.” 


Theranos Response to Statement 3 — D5311: The laboratory has revised its audit 
procedures to establish the procedures by which audits are to be conducted. (Ex. BB, Tab 8). 
Training on these procedures has occurred. (Ex. BB, Tabs 8B-C). The revised procedures 
establish that tracer audits must occur monthly; provide the protocol for a tracer audit; and 
require that tracer audits be documented using the tracer audit form, CL FRM 00046 F2. 

(Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a quarterly audit to 
cover the quality management systems; provide the protocol for quarterly audits; and require 
that quarterly audits are documented using the audit record form, CL QOP 00046 F1. (Ex. 
BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear that “[b]oth 
tracer and quarterly audits are reviewed, analyzed and presented based on findings and 
analysis” during the QMPI Program meetings. (Ex. BB Tab 7§ 7.5). Training on the revised 
QMPI procedures has occurred. (Ex. BB, Tab7B-C). 


D5391 


Finding #1 


D5391 Finding #1 — March 18 Letter Statement 1: “Zhe submission references ‘Ex. A, Tab 
4.’ We found no such reference contained in the materials provided to CMS.” 


Theranos Response to Statement 1 — D5391 Finding #1: The citation reference to Ex. A, 
Tab 4, § 7.1.2.2.d.2 was an inadvertent transcription error and should not have been included. 


D5391 Finding #1 —- March 18 Letter Statement 2: “Zo ensure the deficient practice does not 
recur, the laboratory indicated that quarterly audits will be performed and suggested that the 
audits results would be reviewed within the laboratory’s QMPI Program. However, the 
laboratory did not establish the procedure by which these quarterly audits are to be conducted. 
In the submission, the laboratory indicates that a ‘tracer audit may [emphasis added [in the 
letter]] be used, but did not provided [sic] a protocol for a ‘tracer audit,’ the means by which a 
‘tracer audit’ would be documented, and whether the results of a ‘tracer audit’ would be the 
information reviewed by the QMPI Program.” 


Theranos Response to Statement 2 — D5391 Finding #1: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 Fl. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]oth tracer and quarterly audits are reviewed, analyzed and presented based on 
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findings and analysis” during the QMPI Program meetings. (Ex. BB Tab 7§ 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tab7B-C). 


Finding #2 


D5391 Finding #2 — March 18 Letter Statement 1: “Zhe submission references ‘Ex. A, Tab 
18, § 3.7.1,’ and ‘Ex. A, Tab 23, $§ 7.14, 7.15, 8.6.1, 8.6.4, 8.6.5.” We found no such reference 
contained in the materials provided to CMS.” 


Theranos Response to Statement 1— D5391 Finding #2: The citation references to Ex. A, 
Tab 4, § 3.7.1 and Ex. A, Tab 23, §§ 7.14, 7.15, 8.6.1, 8.6.4, 8.6.5 were inadvertent 
transcription errors and should not have been included. 


D5391 Finding #2 — March 18 Letter Statement 2: “Af the time of the onsite survey, the 
laboratory failed to establish a written protocol for the daily review of patient specimens 
referred to other laboratories for testing to ensure timely receipt and reporting of test results 
performed by other laboratories. The laboratory did not provide such a protocol with its 
submission.” 


Theranos Response to Statement 2 — D5391 Finding #2: The lab has implemented revised 
written procedures for referral testing, which includes the protocol for daily review of patient 
specimens referred to other laboratories for testing to ensure timely receipt and reporting of 
test results performed by other laboratories. (Ex. BB, Tab 10 § 8.7). The revised procedure 
requires written documentation of this daily review. Training on that SOP has occurred. 

(Ex. BB, Tabs 10B-10C). The revised QMPI Program procedures require a review of 
referral testing and turnaround time. (Ex. BB, Tab 7 §§ 7.5.3, 7.5.4). 


D5391 Finding #2 — March 18 Letter Statement 3: “Jn addition, it is unclear how the 
laboratory’s QMPI Program would ensure the establishment of written protocols for all the 
laboratory's processes.” 


Theranos Response to Statement 3 — D5391 Finding #2: The laboratory’s improved 
document control procedures, SOP on SOPs, audit procedures, and QMPI Program ensure 
the laboratory has written protocols for all of the laboratory’s processes. (Ex. BB, Tabs 1-2, 
7-8). The document control procedures identify levels and categories of procedures (Ex. BB, 
Tab 1 §§ 3.11, 6.1); the SOP on SOPs includes procedures and templates to ensure that there 
are written protocols for all processes that should be covered by an SOP (Ex. BB, Tab 2); the 
revised audit procedures include monthly tracer audits that would identify any instance where 
a written protocol did not exist for a process (Ex. BB, Tab 8 § 8.1); and the QMPI Program 
requires review of tracer audit results and of written procedures that are new, revised or in 
process. (Ex. BB, Tab 7§§ 7.1, 7.5). Training on these procedures has occurred. (Ex. BB, 
Tabs 1B-1C, 2B-2C, 7B-7C, 8B-8C). 
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D5393 


D5393 — March 18 Letter Statement 1: “Zhe submission references ‘Ex. A, Tab 23, $$ 7.14, 
7.15, 8.6.1, 8.6.4, 8.6.5.’ We found no such reference contained in the materials provided to 
CMS.” 


Theranos Response to Statement 1 — D5393: The citation references to Ex. A, Tab 23, §§ 
7.14, 7.15, 8.6.1, 8.6.4, 8.6.5 was an inadvertent transcription error and should not have been 
included. 


D5393 — March 18 Letter Statement 2: “A/ the time of the onsite survey, the laboratory failed 
to establish a written protocol for the daily review of patient specimens referred to other 
laboratories for testing to ensure timely receipt and reporting of test results performed by other 
laboratories. The laboratory did not provide such a protocol with its submission.” 


Theranos Response to Statement 2 — D5393 The lab has implemented revised written 
procedures for referral testing, which includes the protocol for daily review of patient 
specimens referred to other laboratories for testing to ensure timely receipt and reporting of 
test results performed by other laboratories. (Ex. BB, Tab 10 § 8.7). The revised procedure 
requires written documentation of this daily review. (Ex. BB, Tab 10 § 8.7, F-1). Training 
on that SOP has occurred. (Ex. BB, Tabs 10B-10C). The revised QMPI Program procedures 
require a review of referral testing and turnaround time. (Ex. BB, Tab 7 §§ 7.5.3, 7.5.4). 


D5393 — March 18 Letter Statement 3: “/n addition, it is unclear how the laboratory’s OMPI 
Program would ensure the establishment of written protocols for all of the laboratory’s 
processes.” 


Theranos Response to Statement 3 — D5393: The laboratory’s improved document control 
procedures, SOP on SOPs, audit procedures, and QMPI Program ensure the laboratory has 
written protocols for all of the laboratory's processes. (Ex. BB, Tabs 1-2, 7-8). The 
document control procedures identify levels and categories of procedures (Ex. BB, Tab 1 §§ 
3.11; 6.1); the SOP on SOPs includes procedures and templates to ensure that there are 
written protocols for all processes that should be covered by an SOP (Ex. BB, Tab 2); the 
revised audit procedures include monthly tracer audits that would identify any instance where 
a written protocol did not exist for a process (Ex. BB, Tab 8 § 8.1); and the QMPI Program 
requires review of tracer audit results and of written procedures that are new, revised or in 
process. (Ex. BB, Tab 7§§ 7.1, 7.5). Training on these procedures has occurred. (Ex. BB, 
Tabs 1B-1C, 2B-2C, 7B-7C, 8B-8C). 


D5393 — March 18 Letter Statement 4: “/n addition, without daily written documentation, it is 
unclear as to how the laboratory's OMPI Program would ensure this quality assessment activity 
has been completed.” 


Theranos Response to Statement 4 — D5393: The laboratory’s revised written procedures 
for referral testing require written documentation of the daily review of referral testing. (Ex. 
BB, Tab 10 § 8.7, F-1). Training on the revised referral testing procedures has occurred. 
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(Ex. BB, Tabs 10B-10C). The revised QMPI procedures require a review of referral testing 
and turnaround time. (Ex. BB, Tab 7 §§ 7.5.3, 7.5.4). Training on the revised QMPI 
procedures has occurred. (Ex. BB, Tabs 7B-7C). 


D5400 


The March 18 Letter states “See our reviews of D5403, D5407, D5413, D5421, D5423, D5429, 
D5437, D5447, D5449, D5469, D5477, D5481, D5775, D5779, D5787, D5791, and D5793.” 
Accordingly, we refer you to our responses to those reviews. 


D5403 


Finding #1 


D5403 Finding #1 — March 18 Letter Statement 1: “Although the laboratory addressed the 
inclusion in its procedure manual of corrective actions to be taken when calibration or quality 
control (OC) results failed to meet the laboratory's criteria for acceptability when using the 
Siemens Advia 2120i instrument, in its submission the laboratory failed to address how it will 
ensure its procedure manuals include applicable components as required by 42 C.F.R. 


493.1251(b).” 


Theranos Response to Statement 1 — D5403 Finding #1: The lab’s revised SOP on SOPs 
includes written protocols and SOP templates, which sets forth protocols for preparing 
procedure manuals that are consistent with 42 C.F.R. § 493.1251(b). (Ex. BB, Tab 2). The 
laboratory’s revised document control policies further ensure that procedure manuals are 
consistent with 42 C.F.R. § 493.1251(b). (Ex. BB, Tab 1 §§ 6.1, 6.2, 6.3). The lab will 
monitor compliance with these procedures and 42 C.F.R. § 493.1251(b) through its monthly 
QMPI meetings, which include a review of written procedure manuals that are new, revised 
or in process. (Ex. BB Tab 7, §§ 7.1, 7.5.6). The lab will also ensure compliance through its 
revised audit procedures of pre-analytic, analytic, and post-analytic practices. (Ex. BB, Tab 
8, §§ 8.1, 8.2). Training on these revised procedures has occurred. (Ex. BB, Tabs 1B-1C; 
2B-2C; 7B-7C; 8B-8C). 


D5403 Finding #1 — March 18 Letter Statement 2: “Zo ensure appropriate and required 
calibration or QC corrective actions were taken, the laboratory indicated that quarterly audits 
will be performed and suggested that the audit results would be reviewed within the laboratory's 
OMPI Program. However, the laboratory did not establish the procedure by which these 
quarterly audits will be performed and suggested that the audits results would be reviewed 
within the laboratory’s OMPI Program. However, the laboratory did not establish the 
procedure by which these quarterly audits are to be conducted. In the submission, the 
laboratory indicates that a ‘tracer audit may [emphasis added] be used, but did not provided 
[sic] a protocol for a ‘tracer audit,’ the means by which a ‘tracer audit’ would be documented, 
and whether the results of a ‘tracer audit’ would be the information reviewed by the OMPI 
Program.” 


Theranos Response to Statement 2 — D5403 Finding #1: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex BB, 
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Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-8C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 F1. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]oth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB, Tab 7 § 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tabs 7B-7C). 


Finding #2: 


D5403 Finding #2 — March 18 Letter Statement 1: “Zhe /aboratory was cited for not having 
a QC procedure for the ‘Edison 3.5 Theranos System’ prior to 5/15/14. The laboratory’s 
submission states that it performed OC before and after 5/15/14; however, it did not address the 
citation. While the data submitted did show that OC was run prior to 5/15/14, the laboratory 
provided no information regarding any investigation as what procedure the laboratory was 
using to determine number, type, frequency, and acceptability criteria.” 


Theranos Response to Statement 1 — D5403 Finding #2: D5403 Finding #2 relates to the 
Theranos Proprietary System 3.5 (“TPS”) The laboratory has performed a review to 
determine number, type, frequency, and acceptability criteria for QC on the TPS 3.5 prior to 
May 15, 2014. Prior to May 15, 2014, the laboratory (i) used the same type of QC material 
as used after May 15, 2014; (ii) the laboratory ran at least two levels of QC prior to May 15, 
2014; (iii) the laboratory was required to pass at least two levels of QC at least 24 hours 
before reporting a patient result; and (iv) the laboratory used the Westgard rules for the 
acceptance criteria. (Ex. FF, Tab 13). 


D5403 Finding #2 — March 18 Letter Statement 2: “Zhe laboratory indicated that systemic 
errors using OC and patient test distribution over time as well as random errors were used to 
evaluate patient impact. Specifically, the laboratory indicated that ‘OC data was reviewed to 
identify >2SD failure and periods of elevated CV. Namely, batches of 10 consecutive OC data 
points were analyzed and the CV was calculated for each QC level during this period. If the CV 
was >30%, then all patient results run during that time period will be voided.’ Ex. E, Tab 1. 
That laboratory did not indicate how ‘periods of elevated CV’ were identified or what time 
periods were reviewed.” 


Theranos Response to Statement 2 — D5403 Finding #2: This deficiency concerns the 
TPS 3.5, which was one of the Theranos proprietary technologies in use in 2014 and 
2015. The laboratory began phasing out the TPS 3.5 in December 2014, and it was fully 
retired in early-August 2015. 


The laboratory values CMS’s feedback, and takes it very seriously. The laboratory has 
conducted a thorough re-review of QC data for each assay run on the TPS 3.5 from January 
1, 2014 until the TPS 3.5 was fully retired in early-August 2015. As explained further in the 
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attached patient assessment analysis, the laboratory is not satisfied with its old quality 
assessment program’s ability to effectively flag and promptly address QC imprecision, QC 
failures, and QC trends on the TPS 3.5. (Ex. AA, Tab 3). The laboratory has new leadership 
and a new QMPI Program with robust quality monitoring and program improvement 
procedures. Based on its re-review of QC data for the TPS 3.5, the laboratory has 
determined that its prior QA program failed to satisfactorily address these types of QC issues 
for assays run on the TPS 3.5 in 2014 and 2015. As corrective action, the laboratory has, out 
of an extreme abundance of caution, voided results reported for assays run on the TPS 3.5 in 
2014 and 2015. (Ex. AA, Tab 3). Many corrected reports have been transmitted, and the 
remainder are being transmitted. (Ex. KK). Transmission will be complete by March 31, 
2016. The remainder of the transmittals and confirmations of receipt will be provided to 
CMS under separate cover. 


D5403 Finding #2 — March 18 Letter Statement 3: “Zo ensure the deficient practice does not 
recur, the laboratory indicated that quarterly audits will be performed and suggested that the 
audits results would be reviewed within the laboratory’s OMPI Program. However, the 
laboratory did not establish the procedure by which these quarterly audits are to be conducted. 
In the submission, the laboratory indicates that a ‘tracer audit may [emphasis added [in the 
letter]] be used, but did not provided [sic] a protocol for a ‘tracer audit,’ the means by which a 
‘tracer audit’ would be documented, and whether the results of a ‘tracer audit’ would be the 
information reviewed by the OMPI Program.” 


Theranos Response to Statement 3 — D5403 Finding #2: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 Fl. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]oth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB Tab 7§ 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tab7B-C). 


D5407 


D5407 — March 18 Letter Statement 1: “A/though the laboratory’s submitted protocol 
(provided at Ex. A, Tab I § 4.13) indicates that the laboratory director must sign all procedures 
prior to use, the response does not include an explanation as to why the cited procedures were 
not signed by the laboratory director prior to use as required by the laboratory’s protocol 
available at the time of the onsite.” 


Theranos Response to Statement 1 — D5407: D5407 states that the following procedures 
showed an effective date in December 2014, but were signed by the lab director on 


14 
TRADE SECRET/CONFIDENTIAL COMMERCIAL INFORMATION 
EXEMPT FROM DISCLOSURE UNDER FREEDOM OF INFORMATION ACT 


FOIA Confidential Treatment Requested by Theranos THER-0534713 
Fed. R. Crim. P. 6(e) material 


Case:228168602 266420) 200d metGERs, Pikenhity/AW2DI, Page 18 of 98 


thera Meo 1701 Page Mill Road P 650.838.9292 theranos.com 


Palo Alto, CA 94304 F 650.838.9165 


September 19, 2015: CL SOP-09102 (ALP),* CL SOP-09161, Revision A (APO), CL SOP- 
09118 (Glucose),* CL SOP-09111 (CO2), CL SOP-10001, Revision B (Measuring 
Prothrombin Time-Innovin (PT)), and CL SOP-15036, Revision A (Edison 3.5 Theranos 
System Daily QC Procedure). 


D5407 also states that CL SOP-06060, Revision A (SensoScientific Monitor) showed an 
effective date of June 23, 2015, but was not signed by the lab director until September 22, 
2015. 


Lastly, D5407 states that CL PLN-14003, Revision A (Master Validation Plan for Routine 
Chemistry Assays on Theranos Devices) showed an effective date of November 4, 2011, but 
was not signed by the lab director under September 19, 2015. To ensure that the record is 
correct with respect to this document, we note that CL PLN-14003, Revision A (Master 
Validation Plan for Routine Chemistry Assays on Theranos Devices), was authored and 
signed by the laboratory’s then-lab director, Dr. Arnold Gelb, on November 4, 2011. (Ex. 
HH, Tab 12). Dr. Gelb remained the Lab Director until January 27, 2013. (Ex. HH, Tab 
4C). Although Dr. Gelb signed CL PLN-14003, the subsequent lab directors did not sign and 
date CL PLN-14003 until Dr. Sunil Dhawan signed it on September 19, 2015. We recognize 
that those prior lab directors should have reviewed, dated and signed CL PLN-14003 before 
that date. 


As reflected by the Lab Director deficiencies identified by CMS, the prior lab directors did 
not provide sufficient oversight over the documents cited in D5407. This appears to be one 
of the reasons why these documents were not timely reviewed and signed by the prior lab 
directors. In addition, the laboratory has found that its quality assurance oversight over these 
documents, including document control oversight, was not sufficient. The laboratory has 
hired a new lab director, who has approved enhanced document control policies and 
procedures to address this issue. (Ex. BB, Tab 1 §§ 5.1, 5.2, 5.3, 6.1, 6.2, 6.3). Training on 
those procedures has occurred. (Ex. BB, Tabs 1B-1C). The laboratory has also hired a 
Document Control Manager whose responsibilities include tracking new and revised policies 
and procedures to ensure that they are reviewed and approved by the lab director before they 
become effective. (Ex. HH, Tab 7C). In addition, the laboratory’s revised QMPI Program 
procedures and agenda require review of procedures that are new, revised, or in process at 
monthly QMPI Program meetings. (Ex. BB, Tab 7 §§ 7.1, 7.5.6). 


415407, Paragraph h, states that the Alkaline Phosphatase SOP that was made effective in 
December 2014 was “CL SOP-81102.” This reference appears to be a typographical error 
because the Alkaline Phosphatase (ALP) SOP that was made effective in December 2014 was 
“CL SOP-09102.” 


> As another point of clarification only, we note that D5407, Paragraph h, states that the glucose 
procedure (CL SOP-09118) was not signed by the LD. That procedure was signed by the lab 
director, but not until September 19, 2015. We recognize that this signature date is still deficient 
under D5407, and provide this information only to clarify the record. 
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D5407 — March 18 Letter Statement 2: “Jn Ex. L, Tab 26 the submission states: “The lab’s 
overarching review of its systems and primary instruments has identified the patients affected or 
having the potential to be affected by this issue.” The laboratory did not define the criteria used 
to make this determination or provide documentation as to how they came to this conclusion.” 


Theranos Response to Statement 2 — D5407: D5407 cites certain documents that were not 
timely signed and approved by the former lab director. 


D5407 cites the follows SOPs for certain assays run on the Siemens Advia XPT because they 
had an effective date in December 2014, but were not signed by the former lab director until 
September 2015: (i) CL SOP-09161, Revision A (Apolipoprotein); (ii) CL SOP-09118 
(Glucose); ° (iii) CL SOP-09111 (CO2); and (iv) CL SOP-09102 (Alkaline Phophatase).” 
The laboratory and its new leadership do not approve of the fact these SOPs were given an 
effective date before Dr. Dhawan signed them. However, there is no potential for these 
untimely signatures by Dr. Dhawan to have affected patients because Dr. Dhawan ultimately 
approved the SOPs without making any modifications or revisions to them. To address other 
deficiencies identified by CMS, the laboratory’s initial submission included a broader 
analysis of QC, proficiency testing results, and patient population distribution data for APO, 
Glucose, CO2, and ALP tests run on the Siemens Advia XPT for the entire period during 
which these SOPs were effective. In response to CMS’s review of D5793 Finding #8, the 
laboratory has extended this analysis to evaluate global quality control indicators. (Ex. AA, 
Tab 11). Based on these analyses, the laboratory has issued corrected reports. (Ex. AA, Tab 
11). The laboratory stopped using the Siemens Advia XPT instrument on November 17, 
2015, and has not used it since then. 


D5407 also cites CL SOP-10001, Revision B (Measuring Prothrombin Time-Innovin (PT)), 
because it was made effective in December 2014, but was not signed by the former lab 
director until September 2015. This SOP concerns PT/INR tests on the BCS XP. Again, the 
laboratory and its new leadership do not approve of the fact this SOP was given an effective 
date before Dr. Dhawan signed the SOP. However, there is no potential for this untimely 


° As a point of clarification only, we note that D5407, Paragraph h, states that the glucose 
procedure (CL SOP-09118) was not signed by the LD. That procedure was signed by the lab 
director, but not until September 19, 2015. We recognize that this signature date is still deficient 
under D5407, and provide this information only to clarify the record. 


7D5407, Paragraph h, states that the Alkaline Phosphatase SOP that was made effective in 
December 2014 was “CL SOP-81102.” This reference appears to be a typographical error 
because the Alkaline Phosphatase (ALP) SOP that was made effective in December 2014 was 
“CL SOP-09102.” 
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signature to have affected patients because Dr. Dhawan ultimately approved the SOP without 
making any modifications or revisions to it. To address other deficiencies identified by 
CMS, the laboratory has performed a broader analysis of all PT/INR tests run on this BCS 
XP instrument, which was first put into use in October 2014. Based on this analysis, which 
included a review of INR calculations, QC data, proficiency testing, and patient population 
distribution data, the laboratory issued corrected reports voiding PT/INR test results reported 
for the period October 2014 through September 2015. (Ex. AA, Tab 7). The laboratory 
stopped using the BCS XP instrument on September 17, 2015, and has not used it since 
then. 


D5407 also cites CL SOP-15036, Revision A (Edison 3.5 Theranos System Daily QC 
Procedure), because it was made effective in December 2014, but was not signed by the 
former lab director until September 2015. This SOP concerns the Theranos Proprietary 
System (TPS), which was being phased-out beginning in December 2014 and was fully 
retired in mid-August 2015. Again, the laboratory and its new leadership do not approve of 
the fact this SOP was given an effective date before Dr. Dhawan signed the SOP to indicate 
his approval of it. However, there is no potential for this untimely signature to have affected 
patients because Dr. Dhawan ultimately approved the SOP without making any modifications 
or revisions to it. To address other deficiencies identified by CMS, the laboratory has 
performed a broad re-analysis of all QC related to tests run on the TPS 3.5 in 2014 and 2015S. 
That analysis is discussed below and is enclosed with this letter at Tab AA, Ex. 3. Based on 
that analysis, the laboratory has, out of an extreme abundance of caution, voided all tests run 
on the TPS 3.5 in 2014 and 2015. Many corrected reports have been transmitted, and the 
remainder are being transmitted. (Ex. KK). Transmission will be complete by March 31, 
2016. The remainder of the transmittals and confirmations of receipt will be provided to 
CMS under separate cover. 


D5407 also cites CL SOP-06060, Revision A (SensoScientific Monitor) because it was made 
effective on June 23, 2015, but was not signed by the former lab director until September 
2015. Again, the laboratory and its new leadership do not approve of the fact this SOP was 
given an effective date before Dr. Dhawan signed the SOP to indicate his approval of it. 
However, there is no potential for this untimely signature to have affected patients because 
Dr. Dhawan ultimately approved the SOP without making any modifications or revisions to 
it. 

Finally, D5407 cites CL PLN-14003, Revision A (Master Validation Plan for Chemistry 
Assays on Theranos Devices). The plan is consistent with 42 C.F.R. § 493.1253(b). CL 
PLN-14003 was authored, approved, signed and dated by Dr. Arnold Gelb. Dr. Gelb was the 
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Laboratory Director when he approved CL PLN-14003.* (Ex. HH, Tabs 4B, 4C, & 12). 
Although Dr. Dhawan should have reviewed the plan sooner, he ultimately approved of it. 
Because Dr. Dhawan approved of the plan without making any changes to it, no patients 
have the potential to have been affected by the finding in D5407, Paragraph g. 


D5413 
Finding #1 


D5413 Finding #1 — March 18 Letter Statement 1: “A/though the laboratory’s submitted 
protocol indicates that the allowable temperature should be posted on the unit (see Ex. A, Tab 
29), the response does not address the incorrect labeling for acceptable temperatures on the 
freezer doors. Documentation contained in Ex. A. Tab 29, § 4.5 indicated that the laboratory 
supervisor or designee was responsible for monitoring and recording temperatures. However, 
we could not determine if the new procedure had been effectuated as no documentation was 
submitted to show that the freezer units had been labeled appropriately.” 


Theranos Response to Statement 1 — D5413 Finding #1: The new procedure has been 
effectuated as reflected in the enclosed documentation showing that the freezer units have 
been labeled appropriately. (Ex. Ll, Tab 2). 


D5413 Finding #1 — March 18 Letter Statement 2: “/n addition, the laboratory states that 
training had occurred subsequent to the survey, but it is not clear who should be trained as 
‘laboratory supervisor or designee’ was not defined.” 


Theranos Response to Statement 2 — D5413 Finding #1: The laboratory has revised this 
procedure to define who should be trained. The revised procedure is enclosed at Ex. BB, Tab 
13. 


D5413 Finding #1 — March 18 Letter Statement 3: “7raining records provided in Ex. A, Tab 
30 included only training documents for one general supervisor.” 


Theranos Response to Statement 3 — D5413: The laboratory has enclosed with this letter a 
complete set of training records on this SOP, which shows that the relevant laboratory 
personnel have been trained. (Ex. BB, Tabs 13B-13C). 


D5413 Finding #1 — March 18 Letter Statement 4: “Several freezers were identified as “not 
used in patient testing” (see Ex. N, Tab A); however, based on interviews during the onsite visit, 


8 At the time that Dr. Gelb authored this plan, his qualifications included, among other things, a 
Master of Science in Materials Science and Engineering (Biomedical); a Fellowship in MIT’s 
Medical Engineering Graduate Program; a Doctor of Medicine from Stanford University; an 
internship in Internal Medicine at the University of Chicago Hospitals; a Fellowship in 
Immunodiagnosis and Surgical Pathology at Stanford University; and a Fellowship in 
Hematopathology at the University of California San Francisco. (Ex. HH, Tab 4A). 
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these freezers were identified as being used for CLIA activities. The submission did not include 
a response or data related to the freezers identified as ‘not used in patient testing.’” 


Theranos Response to Statement 4 — D5413 Finding #1: D5413 identifies six -20C 
freezers and four -80C freezers. In the lab’s submission, we explain that five of those 
freezers did not contain materials that would be used in the future for clinical patient testing. 
Rather, they only contained material that might be used in the future for research and 
development. 


Based on CMS’s March 18 letter, it appears that there was a miscommunication when the 
CMS surveyors were onsite. D5413, Paragraph f, states that CMS had a discussion about 
these freezers with the “Director of Assay Systems and technical supervisor at 11/19/15 at 
approximately 11 am.” The Director of Assay Systems did not have any regular involvement 
with those freezers at that time and did not have knowledge of their contents. The “technical 
supervisor” in attendance on November 19, 2015 joined the company on October 26, 2015; 
he also did not have regular involvement with those freezers at the time and did not have 
knowledge of their contents. (Ex. HH, Tab 8). To the extent CMS understood either of these 
individuals to be confirming the contents of these freezers or whether these freezers were 
used in clinical patient testing, there was a miscommunication. 


The laboratory has looked into this issue further, and has reconfirmed that the statements in 
the laboratory’s submission are accurate. As reflected in the enclosed documentation: 


e On November 19, 2015, there were four -20 C freezers in the microbiology or BUGS 
laboratory. 7061 BUGS -20 C Freezer 4 did not contain any materials for future use 
in clinical patient testing. (Ex. II, Tab 3B). 


e On November 19, 2015, there was one -20 C freezer in the Normandy laboratory. 
Clinical patient testing in the Normandy laboratory stopped on September 20, 2015, 
and there has not been any clinical patient testing in the Normandy laboratory since 
then. The 1 Normandy -20 C freezer did not contain any materials for future use in 
patient testing. (Ex. II, Tab 3C). 


e On November 19, 2015, there were four -80 C freezers. 7098 -80 C Freezer 1 Nuair, 
7111 -80 C Freezer 2 Thermo, and 7113 -80 2 CLIA Lab did not contain any 
materials for future use in clinical patient testing. While named 7113 -80 2 CLIA 
Lab, it was only used to store old specimens that were not for use in clinical patient 
testing. (Ex. II, Tab 3A). 


D5413 Finding #1 — March 18 Letter Statement 5: “7o ensure the deficient practice does not 
recur, the laboratory indicated that quarterly audits will be performed and suggested that the 
audits results would be reviewed within the laboratory’s OMPI Program. However, the 
laboratory did not establish the procedure by which these quarterly audits are to be conducted. 
In the submission, the laboratory indicates that a ‘tracer audit may [emphasis added [in the 
letter]] be used, but did not provided [sic] a protocol for a ‘tracer audit,’ the means by which a 
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‘tracer audit’ would be documented, and whether the results of a ‘tracer audit’ would be the 
information reviewed by the OMPI Program.” 


Theranos Response to Statement 5 — D5413 Finding #1: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 Fl. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]Joth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB Tab 7§ 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tab7B-C). 


Finding #2 


D5413 Finding #2 — March 18 Letter Statement 1: “Zhe submission references ‘Ex. I, Tabs 2- 
6.’ We located these tabs, but found no documentation in Tabs 2, 5, and 6.” 


Theranos Response to Statement 1 — D5413 Finding #2: There was an inadvertent 
copying error for Exhibit I to the initial submission. The laboratory has enclosed the 
applicable documents with this letter, as well as additional documents related to the 
laboratory’s broader analysis of the PT/INR test over the entire time it was run on this 
Siemens BCS XP instrument. (Ex. GG). 


D5413 Finding #2 — March 18 Letter Statement 2: “A/though the laboratory’s submitted 
protocol indicates that the laboratory was required to check manufacturer package inserts prior 
to use (see Ex. A, Tab 31, $8.1.2), which was also stated in the laboratory’s written procedure 
available prior to the onsite survey, the laboratory provided no documentation showing that this 
protocol has been effectuated and no indication that package inserts for new lot numbers have 
been checked.” 


Theranos Response to Statement 2 — D5413 Finding #2: D5413 concerns the laboratory’s 
failure to follow the manufacturer’s instructions for the expiration date of one lot of Innovin 
(thromboplastin) used for Prothrombin Time/International Normalized Ratio (PT/INR) 
testing on the Siemens BCS XP. The laboratory stopped running tests on Siemens BCS XP 
on September 17, 2015, and has not run any tests on the Siemens BCS XP since then. 


Although the laboratory is not running PT/INR tests on the Siemens BCS XP, we have 
enclosed documentation related to a different test that is in use, which shows that the 
laboratory has effectuated its protocol to check manufacturer’s instructions for the expiration 
date. This documentation includes a photograph of the manufacturer's package insert for 
IRISpec and a photograph of the bottles showing a handwritten open date and expiration date 
that is consistent with the 15-day “open stability” specification contained in the package 
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insert . (Ex. HH, Tabs 11A-11B). As these photographs show, the laboratory effectuated its 
protocol by reviewing the manufacturer’s instructions, initialed the material, and wrote the 
correct expiration date on the bottle. (Ex. HH, Tab 11A). 


D5413 Finding #2 — March 18 Letter Statement 3: “Jn addition, the submission states that 
“the lab has conducted training on those procedures;” however, the training documents 
submitted in Ex. A, Tab 32 do not include any training specific to the cited deficiency. We were 
unable to verify that training occurred as stated.” 


Theranos Response to Statement 3 — D5413 Finding #2: D5413 Finding #2 states that the 
laboratory “failed to follow the manufacturer instructions for expiration date of Innovin 
(thromboplastin) used for Prothrombin Time/International Normalized Mean Ratio (PT/INR) 
testing.” The laboratory’s submission included records for training on the revised reagent 
qualification and management procedures. (Ex. BB, Tab 14B). The training addressed 
review of manufacturer specification and package inserts. (Ex. BB, Tab 14). The laboratory 
has performed supplemental training that specifically focuses on reviewing and following 
manufacturer instructions for expiration dates. (Ex. BB, Tab 14D). That training used this 
deficiency about Innovin (thromboplastin) as a case study. (Ex. BB, Tab 14D). 


D5413 Finding #2 — March 18 Letter Statement 4: “Jn Ex. [, Tab 1, the laboratory provided 
lists of patient specimen accession numbers for which the laboratory issued corrected reports; 
however, the laboratory did not provide copies of the corrected reports.” 


Theranos Response to Statement 4 — D5413 Finding #2: To address this and other 
deficiencies related to PT/INR, the laboratory conducted a further patient assessment analysis 
that covers the entire period during which this Siemens BCS XP instrument was in use in the 
lab. (Ex. AA, Tab 7). As a result of that analysis, the laboratory has voided all results for 
PT/INR from October 2014 through September 2015. (Ex. AA, Tab 7). Many corrected 
reports have been transmitted, and the remainder are being transmitted. (Ex. KK; Ex. GG, 
Tab 7). Transmission will be complete by March 31, 2016. The remainder of the 
transmittals and confirmations of receipt will be provided to CMS under separate cover. 


D5421 
Finding #1 


D5421 Finding #1— March 18 Letter Statement 1: 7he submission references “Ex. A, Tab 9, 
$7.2.4.4 and Ex. B, Tabs 46-50, 52-56, 57-61.” We found no such references contained in the 
materials provided to CMS.” 


Theranos Response to Statement 1 — D5421 Finding #1: There was an inadvertent 
copying error for Exhibit I to the initial submission. The laboratory has enclosed the 
applicable documents with this letter, as well as additional documents related to the 
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laboratory’s broader analysis of the PT/INR test over the entire time it was run on this 
Siemens BCS XP instrument. (Ex. GG). 


D5421 Finding #1— March 18 Letter Statement 2: “Zhe laboratory's submitted protocol (Ex. A, 
Tab 9, § 4.2) states: ‘If the verification is performed by the vendor, the laboratory is involved in 
all aspects of the verification and the final verification documentation is reviewed and signed off 
by the Laboratory Director.’ We were unable to determine what specific aspects of the method 
verification the laboratory would be performing or if vendors would continue to perform method 
verifications. ” 


Theranos Response to Statement 2 — D5421 Finding #1: The laboratory has further 
revised its method verification SOP to clarify that the procedure requires the laboratory to 
perform method verifications. To be clear, the vendor will not continue to perform method 
verifications. (Ex. BB, Tab 5 § 4.1.2). 


D5421 Finding #1— March 18 Letter Statement 2: “Zhe submitted protocol included form CL 
FRM-00022-F 1, Verification Results (Ex. A, Tab 9), which appears to be pre-signed by the 
laboratory director. We were unable to determine if the laboratory director will “review and 
sign off’ on each method verification.” 


Theranos Response to Statement 2 — D5421 Finding #1: The copy of CL FRM-00022-F1 
included with the submission was not a pre-signed form. Rather, the laboratory director 
signed the version of the form attached to the revised method verification SOP simply to 
indicate that he approved of it as a new form that could be used. To eliminate any confusion 
or potential confusion caused by this way of approving forms, we have revised the lab’s 
document control policy to make clear that forms attached to an SOP are part of the SOP, and 
therefore are approved when the lab director approves the SOP. (Ex. BB, Tab 1). Consistent 
with this revision, the enclosed copy of CL FRM-00022-F | no longer has any signatures on 
it. (Ex. BB, Tab 5 F1). 


D5421 Finding #1— March 18 Letter Statement 3: “We note that in the laboratory’s 
submission related to test performance specifications for the Advia XPT instrument, it was the 
laboratory’s policy to accept %CV values that were up to 1.5 times the %CV values stated by the 
instrument manufacturer for any given chemistry analyte. In the submission, we found no 
explanation as to why 1.5 times the manufacturer’s %CV was acceptable to the laboratory. We 
also note that the submitted re-verification of test performance specifications did not follow the 
submitted protocol for test verification; therefore, we could not determine if the re-verifications 
were adequate or how the laboratory determined if patients were affected or potentially 
affected.” 


Theranos Response to Statement 3 — D5421 Finding #1: Prior to the survey, the 
laboratory did not employ a policy to accept %CV values that were up to 1.5 times the %CV 
values stated by the manufacturer, and the verification reports that CMS reviewed did not use 
1.5 times the %CV stated by the manufacturer. Likewise, the laboratory’s revised method 
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verification procedure does not include a 1.5 times %CV policy. (Ex. BB, Tab 5).° Rather, 
it requires that the laboratory verify the manufacturer’s imprecision claim. (Ex. BB, Tab 5 § 
8.2). Training on the revised procedure has occurred. (Ex. BB, Tabs SB-5C). Additionally, 
there appears to have been a miscommunication because the laboratory stopped running tests 
on the Siemens Advia XPT on November 17, 2015, and has not run any tests on the 
instrument since then. Accordingly, the laboratory has not re-verified any assay on the 
Siemens Advia XPT. References to method verification data in documents enclosed with the 
submission are to verification reports prepared before the survey began and before the 
laboratory’s revised procedures were made effective (those reports were either reviewed by 
CMS or available to CMS during the survey), which were referenced for context. 


Finding #2 


D5421 Finding #2— March 18 Letter Statement 1: “A/though the laboratory’s submitted 
protocol (Ex. A, Tab 9) included a method verification procedure which addressed verification of 
accuracy, precision, and reportable range, the laboratory provided no documentation showing 
that this protocol had been followed to re-verify performance specifications. The submission 
also stated: Before the lab resumes any test on the Advia XPT, the lab will ensure that the test 
has been re-verified pursuant to the lab’s improved method verification procedures that have 
been approved by the laboratory director.” The completion date for the correction of this 
deficiency was indicated on the submitted Allegation of Compliance as February 12, 2016, 
which, in order to determine correction, should include the re-verification documentation for the 
Advia XPT. However, the laboratory provided no documentation that the re-verification had 
been performed. We note that in the laboratory’s submission related to test performance 
specifications for the Advia XPT instrument it was the laboratory’s policy to accept %CV values 
that were up to 1.5 times the %CV values stated by the instrument manufacturer for any given 
chemistry analyte. In the submission, we found no explanation as to why 1.5 times the 
manufacturer's %CV was acceptable to the laboratory. 


Theranos Response to Statement 1 — D5421 Finding #2: Prior to the survey, the 
laboratory did not employ a policy to accept %CV values that were up to 1.5 times the %CV 
values stated by the manufacturer, and the verification reports that CMS reviewed did not use 
1.5 times the %CV stated by the manufacturer. Likewise, the laboratory’s revised method 
verification procedure does not include a 1.5 times %CV policy. (Ex. BB, Tab 5).!° Rather, 
it requires that the laboratory verify the manufacturer’s imprecision claim. (Ex. BB, Tab 5 § 


° The laboratory referred to a 1.5 times %CV in its response based upon Bio-Rad’s coefficient of 
variation rate. (http://www.qcnet.com/Portals/50/PDFs/QCW orkbook2008 _Jun08.pdf). Upon 
further review, the laboratory does not believe that Bio-Rad’s coefficient of variation rate is 
applicable here. 


'0 The laboratory referred to a 1.5 times %CV in its response based upon Bio-Rad’s coefficient 
of variation rate. (http://www.qcnet.com/Portals/50/PDFs/QCWorkbook2008_Jun08 pdf). Upon 
further review, the laboratory does not believe that Bio-Rad’s coefficient of variation rate is 
applicable here. 
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8.2). Training on the revised procedure has occurred. (Ex. BB, Tabs 5B-SC). Additionally, 
there appears to have been a miscommunication because the laboratory stopped running tests 
on the Siemens Advia XPT on November 17, 2015, and has not run any tests on the 
instrument since then. Accordingly, the laboratory has not re-verified any assay on the 
Siemens Advia XPT. References to method verification data in documents enclosed with the 
submission are to verification reports prepared before the survey began and before the 
laboratory’s revised procedures were made effective (those reports were either reviewed by 
CMS or available to CMS during the survey), which were referenced for context. 


D5421 Finding #2— March 18 Letter Statement 2: “We also found no explanation as how 
“normal patient distribution” related to determining if patients were affected or potentially 
affected.” 


Theranos Response to Statement 2 — D5421 Finding #2: The laboratory has enclosed a 
revised analysis to clarify its examination of these patient test result distributions, which 
should eliminate the confusion created by the phrase “normal patient distribution.” (Ex. AA, 
Tab 5). 


D5421 Finding #2— March 18 Letter Statement 3: “/n addition, the laboratory's submission 
(Ex. B, various analyte tabs) states: “Matrix comparisons done by the manufacturer between 
serum and plasma showed correlation.” No documentation was submitted supporting this 
statement.” 


Theranos Response to Statement 3 — D5421 Finding #2: The laboratory has enclosed with 
this letter documentation supporting that matrix comparisons done by the manufacturer 
between serum and plasma showed correlation. (Ex. HH, Tab 13). 


D5421 Finding #2— March 18 Letter Statement 4: “Zhe submission also states that “the lab 
has conducted training on these procedures to ensure that the practice is consistent with them,” 
but the laboratory did not include complete training documentation to support this assertion.” 


Theranos Response to Statement 4 — D5421 Finding #2: The laboratory’s submission 
included training documentation for seven laboratory personnel. The laboratory has enclosed 
with this letter additional training documentation for the lab’s method verification 
procedures. (Ex. BB, Tabs 5B-SC). 


D5421 Finding #2— March 18 Letter Statement 5: “Because the laboratory has not shown 
whether it can follow its own validation protocols, it was uncertain whether the laboratory’s 
quality assessment mechanisms could monitor the laboratory’s corrective actions and ensure this 
deficient practice does not recur.” 


Theranos Response to Statement 5 — D5421 Finding #2: As stated above, there appears to 
have been a miscommunication because the laboratory stopped running tests on the Siemens 


24 
TRADE SECRET/CONFIDENTIAL COMMERCIAL INFORMATION 
EXEMPT FROM DISCLOSURE UNDER FREEDOM OF INFORMATION ACT 


FOIA Confidential Treatment Requested by Theranos THER-0534723 
Fed. R. Crim. P. 6(e) material 


Case:228168602 266420) 200d metGEws, Pikenhity/AW2D, Page 26 of 98 


thera Meo 1701 Page Mill Road P 650.838.9292 theranos.com 


Palo Alto, CA 94304 F 650.838.9165 


Advia XPT on November 17, 2015, and has not run any tests on the instrument since then. 
Accordingly, the laboratory has not re-verified any assay on the Siemens Advia XPT. 


The laboratory is capable of ensuring that the deficient practice in D5421 does not recur. 
Significantly, the laboratory has brought in an entirely new leadership team that is directly 
involved with implementing and monitoring the laboratory’s new procedures. The new 
laboratory director, Dr. Kingshuk Das, began working for the laboratory full-time as of 
March 14, 2016. Dr. Das is also one of the laboratory’s technical supervisors, which further 
reflects his level of involvement in the laboratory. In addition, the laboratory has hired 
Donald Tschirhart, M.D to be a full-time co-director (under California regulations) and a 
full-time clinical consultant. Dr. Tschirhart has over fifteen years of experience as a clinical 
laboratory director, and is a valuable part of the laboratory’s new leadership and oversight 
team. The laboratory’s new leadership team has installed an entirely redesigned QMPI 
Program, as well as robust audit procedures that required monthly and quarterly audits. 


D5423 


D5423 — March 18 Letter Statement 1: “A/though the laboratory’s submitted protocol (Lx. A, 
Tab 9) included a method verification procedure which addressed verification of accuracy, 
precision, and reportable range, the laboratory provided no documentation showing that this 
protocol had been followed to re-verify performance specifications. The submission also stated: 
Before the lab resumes any test on the Advia XPT, the lab will ensure that the test has been re- 
verified pursuant to the lab’s improved method verification procedures that have been approved 
by the laboratory director.” The completion date for the correction of this deficiency was 
indicated on the submitted Allegation of Compliance as February 12, 2016, which, in order to 
determine correction, should include the re-verification documentation for the Advia XPT. 
However, the laboratory provided no documentation that the re-verification had been performed. 
We note that in the laboratory’s submission related to test performance specifications for the 
Advia XPT instrument, it was the laboratory’s policy to accept %CV values that were up to 1.5 
times the %CV values stated by the instrument manufacturer for any given chemistry analyte. In 
the submission, we found no explanation as to why 1.5 times the manufacturer ’s %CV was 
acceptable to the laboratory.” 


Theranos Response to Statement 1 — D5423: Prior to the survey, the laboratory did not 
employ a policy to accept %CV values that were up to 1.5 times the %CV values stated by 
the manufacturer, and the verification reports that CMS reviewed did not use 1.5 times the 
%CV stated by the manufacturer. Likewise, the laboratory’s revised method verification 
procedure does not include a 1.5 times %CV policy. (Ex. BB, Tab 5).'' Rather, it requires 


'! The laboratory referred to a 1.5 times %CV in its response based upon Bio-Rad’s coefficient 
of variation rate. (http://www.qcnet.com/Portals/50/PDFs/QCWorkbook2008 Jun08.pdf). Upon 
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that the laboratory verify the manufacturer’s imprecision claim. (Ex. BB, Tab 5 § 8.2). 
Training on the revised procedure has occurred. (Ex. BB, Tabs SB-5C). Additionally, there 
appears to have been a miscommunication because the laboratory stopped running tests on 
the Siemens Advia XPT on November 17, 2015, and has not run any tests on the instrument 
since then. Accordingly, the laboratory has not re-verified any assay on the Siemens Advia 
XPT. References to method verification data in documents enclosed with the submission are 
to verification reports prepared before the survey began and before the laboratory’s revised 
procedures were made effective (those reports were either reviewed by CMS or available to 
CMS during the survey), which were referenced for context. 


D5423 — March 18 Letter Statement 2: “We also found no explanation as how “normal 
patient distribution” relates to determining if patients were affected or potentially affected.” 


Theranos Response to Statement 2 — D5423: The laboratory has enclosed a revised 
analysis to clarify its examination of these patient test result distributions, which should 
eliminate the confusion created by the phrase “normal patient distribution.” (Ex. AA, Tab 5). 


D5423 — March 18 Letter Statement 3: ‘/he /aboratory’s procedure as well as the method 
verification for alkaline phosphatase was reviewed at the onsite survey and the reportable range 
was documented as 0 - 1100 IU/L thus modifying the test. At the time of survey, the laboratory 
stated that the reportable range was 10 - 1100 IU/L per the manufacturer, and that the 
reportable range of 0 - 1100 IU/L was an error. The laboratory confirmed that the lowest 
reportable value established by the manufacturer was 10 IU/L. The Patient Impact Analysis 
submitted by the laboratory at Ex. B, Tab 6 now indicates that the reportable range should be 5 - 
1100 IU/L which is still lower than the range established by the manufacturer; however, the 
laboratory did not submit any documentation to support this new reportable range.” 


Theranos Response to Statement 2 — D5423: The “Patient Impact Analysis” submitted by 
the laboratory at Ex. B, Tab 6 of its submission included a typographical error. As reflected 
in the revised patient assessment enclosed with this letter, the reportable range should be 12 — 
909 U/L. (Ex. AA, Tab 6). 


D5423 — March 18 Letter Statement 4; “Zhe Patient Impact Analysis also indicates that 
review of the PT showed a negative bias for PT samples CAP C-A 2015 and CAP-B 2015, but 
the laboratory determined that no corrected were needed. The submission also states: “There 
were no significant trends or bias in patient population data.” The laboratory provided no 
explanations for these assertions.” 


further review, the laboratory does not believe that Bio-Rad’s coefficient of variation rate is 
applicable here. 
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Theranos Response to Statement 4 — D5423: The laboratory’s patient assessment analysis 
appears to have included an inadvertent copy and paste mistake. The laboratory has enclosed 
its revised analysis. (Ex. AA, Tab 2). 


D5423 — March 18 Letter Statement 5: “Jn addition, the laboratory’s submission states at Ex. 
B, Tab 6: “Matrix comparisons done by the manufacturer between serum and plasma showed 
correlation.” The laboratory submitted no documentation supporting this statement.” 


Theranos Response to Statement 5 — D5423: The laboratory has enclosed with this letter 
documentation supporting that matrix comparisons done by the manufacturer between serum 
and plasma showed correlation. (Ex. HH, Tab 13) 


D5423 — March 18 Letter Statement 6: “Zhe submission also states that “the lab has 
conducted training on these procedures to ensure that the practice is consistent with them,” but 
did not include complete training documentation.” 


Theranos Response to Statement 6 — D5423: The laboratory’s submission included 
training documentation on the method verification procedures for seven laboratory 
personnel. The laboratory has enclosed with this letter additional training documentation 
for the lab’s method verification procedures. (Ex. BB, Tabs 5B-5C). 


D5423 — March 18 Letter Statement 7: “Because the laboratory has not shown whether it 
can follow its own validation protocols, it is uncertain whether the laboratory’s quality 
assessment mechanisms could monitor the laboratory’s corrective actions and ensure this 
deficient practice does not recur.” 


Theranos Response to Statement 7 — D5423: As stated above, there appears to have 
been a miscommunication because the laboratory stopped running tests on the Siemens 
Advia XPT on November 17, 2015, and has not run any tests on the instrument since 
then. Accordingly, the laboratory has not re-verified any assay on the Siemens Advia 
XPT. 


The laboratory is capable of ensuring that the deficient practice in D5423 does not recur. 
Significantly, the laboratory has brought in an entirely new leadership team that is 
directly involved with implementing and monitoring the laboratory’s new procedures. 
The new laboratory director, Dr. Kingshuk Das, began working for the laboratory full- 
time as of March 14, 2016. Dr. Das is also one of the laboratory’s technical supervisors, 
which further reflects his level of involvement in the laboratory. In addition, the 
laboratory has hired Donald Tschirhart, M.D to be a full-time co-director (under 
California regulations) and a full-time clinical consultant. Dr. Tschirhart has over fifteen 
years of experience as a clinical laboratory director, and is a valuable part of the 
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laboratory’s new leadership and oversight team. The laboratory’s new leadership team 
has installed an entirely redesigned QMPI Program, as well as robust audit procedures 
that required monthly and quarterly audits. 


D5429 


D5429 — March 18 Letter Statement: “Zo ensure the deficient practice does not recur, the 
laboratory indicated that quarterly audits will be performed and suggested that the audits results 
would be reviewed within the laboratory’s OMPI Program. However, the laboratory did not 
establish the procedure by which these quarterly audits are to be conducted. In the submission, 
the laboratory indicates that a ‘tracer audit may [emphasis added [in the letter]] be used, but 
did not provided [sic] a protocol for a ‘tracer audit,’ the means by which a ‘tracer audit’ would 
be documented, and whether the results of a ‘tracer audit’ would be the information reviewed by 
the OMPI Program.” 


Theranos Response to Statement — D5429; The laboratory has revised its audit procedures 
to establish the procedures by which audits are to be conducted. (Ex. BB, Tab 8). Training 
on these procedures has occurred. (Ex. BB, Tabs 8B-C). The revised procedures establish 
that tracer audits must occur monthly; provide the protocol for a tracer audit; and require that 
tracer audits be documented using the tracer audit form, CL FRM 00046 F2. (Ex. BB, Tab 8 
§ 8.1). In addition, the revised audit procedures require a quarterly audit to cover the quality 
management systems; provide the protocol for quarterly audits; and require that quarterly 
audits are documented using the audit record form, CL QOP 00046 F1. (Ex. BB, Tab 8 § 
8.2). The QMPI Program QOP has been revised to make clear that “[b]oth tracer and 
quarterly audits are reviewed, analyzed and presented based on findings and analysis” during 
the QMPI Program meetings. (Ex. BB Tab 7§ 7.5). Training on the revised QMPI 
procedures has occurred. (Ex. BB, Tab7B-C). 


D3437 


Findings #1 & #2 


D5437 Findings #1 & #2 — March 18 Letter Statement 1: “A/ the time of the onsite survey, the 
laboratory failed to maintain calibration documentation for the complete blood count (CBC) 
instruments. In the submission, we found no evidence the laboratory has re-established its 
calibration documentation for the CBC instruments.” 


Theranos Response to Statement 1 — D5437 Findings #1 & #2: There appears to have 
been a miscommunication. The laboratory stopped using its Siemens Advia 21201 
instruments for clinical patient testing on November 17, 2015, and has not used them since 
then. The two Drew-3 instruments were retired before the survey began on September 22, 
2015 and before the laboratory’s revised procedures were made effective. Because the 
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laboratory’s “CBC instruments” are not in use, the laboratory has not “re-established” 
calibration documentation for those instruments. 


D5437 Findings #1 & #2 — March 18 Letter Statement 2: “Jn Ex. F, the laboratory provided 
lists of patient specimen accession numbers for which the laboratory intended to issue corrected 
test reports; however, the laboratory provided no documentation to indicate corrected reports 
were generated and issued.” 


Theranos Response to Statement 2 — D5437 Findings #1 & #2: Many corrected reports 
have been transmitted, and the remainder are being transmitted. (Ex. KK). Transmission will 
be complete by March 31, 2016. The remainder of the transmittals and confirmations of 
receipt will be provided to CMS under separate cover. 


D5437 Findings #1 & #2 — March 18 Letter Statement 3: “Zo ensure appropriate and 
required calibration or OC corrective actions were taken, the laboratory indicated that quarterly 
audits will be performed and suggested that the audit results would be reviewed within the 
laboratory’s OMPI Program. However, the laboratory did not establish the procedure by which 
these quarterly audits will be performed and suggested that the audits results would be reviewed 
within the laboratory's OMPI Program. However, the laboratory did not establish the 
procedure by which these quarterly audits are to be conducted. In the submission, the 
laboratory indicates that a ‘tracer audit may [emphasis added] be used, but did not provided 
[sic] a protocol for a ‘tracer audit,’ the means by which a ‘tracer audit’ would be documented, 
and whether the results of a ‘tracer audit’ would be the information reviewed by the QMPI 
Program.” 


Theranos Response to Statement 3 — D5437 Finding #1 & #2: 


The laboratory has revised its audit procedures to establish the procedures by which audits 
are to be conducted. (Ex. BB, Tab 8). Training on these procedures has occurred. (Ex. BB, 
Tabs 8B-C). The revised procedures establish that tracer audits must occur monthly; provide 
the protocol for a tracer audit; and require that tracer audits be documented using the tracer 
audit form, CL FRM 00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit 
procedures require a quarterly audit to cover the quality management systems; provide the 
protocol for quarterly audits; and require that quarterly audits are documented using the audit 
record form, CL QOP 00046 F1. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been 
revised to make clear that “[b]oth tracer and quarterly audits are reviewed, analyzed and 
presented based on findings and analysis” during the QMPI Program meetings. (Ex. BB Tab 
7§ 7.5). Training on the revised QMPI procedures has occurred. (Ex. BB, Tab7B-C). 


D5437 Findings #1 & #2 — March 18 Letter Statement 4: “Jn addition, we note that in the 
laboratory’s re-verification of test performance specifications for the CBC instruments, it was 
the laboratory’s policy to accept %CV values that were up to 1.5 times the %CV values stated by 
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the instrument manufacturer for any given CBC analyte. In the submission, we find no 
explanation as to why 1.5 times the manufacturer’s %CV is acceptable to the laboratory.” 


Theranos Response to Statement 4 — D5437 Finding #1 & #2: Prior to the survey, the 
laboratory did not employ a policy to accept %CV values that were up to 1.5 times the %CV 
values stated by the manufacturer, and the verification reports that CMS reviewed did not use 
1.5 times the %CV stated by the manufacturer. Likewise, the laboratory’s revised method 
verification procedure does not include a 1.5 times %CV policy. (Ex. BB, Tab 5).!? Rather, 
it requires that the laboratory verify the manufacturer’s imprecision claim. (Ex. BB, Tab 5 § 
8.2). Training on the revised procedure has occurred. (Ex. BB, Tabs 5B-S5C). Additionally, 
there appears to have been a miscommunication because the laboratory stopped running tests 
on the Siemens Advia 2120i instruments on November 17, 2015, and has not run any tests on 
the instruments since then. The laboratory has retired its two Drew-3 instruments. 
Accordingly, the laboratory has not re-verified any assay on “the CBC instruments.” 
References to method verification data in documents enclosed with the submission are to 
verification reports prepared before the survey began and before the laboratory’s revised 
procedures were made effective (those reports were either reviewed by CMS or available to 
CMS during the survey), which were referenced for context. 


D5447 


D5447 — March 18 Letter Statement 1: “A/though the laboratory's submitted protocol 
indicates that a two-level OC slide review is now required every day of patient testing when 
using the Cellavision, the laboratory provided no documentation of or information about the OC 
materials used, how the statistical parameters of the QC materials would be determined, how 
QC test results will be documented, and whether laboratory staff has been trained on this new 
protocol.” 


Theranos Response to Statement 1 — D5447: The laboratory has revised the Cellavision 
SOP to include information about the QC materials used, how the statistical parameters of 
the QC materials would be determined, and how QC test results will be documented. (Ex. 
BB, Tab 15 ). However, this SOP is not presently in use, because the laboratory stopped 
using the Cellavision on November 17, 2015, and has not used the Cellavision since then. 
Because the Cellavision is not in use, the laboratory has not conducted a training on this 
revised (but inactive) SOP. 


'? The laboratory referred to a 1.5 times %CV in its response based upon Bio-Rad’s coefficient 
of variation rate. (http://www.qcnet.com/Portals/50/PDFs/QCWorkbook2008_Jun08 pdf). Upon 
further review, the laboratory does not believe that Bio-Rad’s coefficient of variation rate is 
applicable here. 
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D5447 — March 18 Letter Statement 2: “Zo ensure the deficient practice does not recur, the 
laboratory indicated that quarterly audits will be performed and suggested that the audits results 
would be reviewed within the laboratory’s OMPI Program. However, the laboratory did not 
establish the procedure by which these quarterly audits are to be conducted. In the submission, 
the laboratory indicates that a ‘tracer audit may [emphasis added [in the letter]] be used, but 
did not provided [sic] a protocol for a ‘tracer audit,’ the means by which a ‘tracer audit’ would 
be documented, and whether the results of a ‘tracer audit’ would be the information reviewed by 
the OMPI Program.” 


Theranos Response to Statement 2 — D5447: The laboratory has revised its audit 
procedures to establish the procedures by which audits are to be conducted. (Ex. BB, Tab 8). 
Training on these procedures has occurred. (Ex. BB, Tabs 8B-C). The revised procedures 
establish that tracer audits must occur monthly; provide the protocol for a tracer audit; and 
require that tracer audits be documented using the tracer audit form, CL FRM 00046 F2. 

(Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a quarterly audit to 
cover the quality management systems; provide the protocol for quarterly audits; and require 
that quarterly audits are documented using the audit record form, CL QOP 00046 F1. (Ex. 
BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear that “[b]oth 
tracer and quarterly audits are reviewed, analyzed and presented based on findings and 
analysis” during the QMPI Program meetings. (Ex. BB Tab 7§ 7.5). Training on the revised 
QMPI procedures has occurred. (Ex. BB, Tab7B-C). 


D5449 


D5449 — March 18 Letter Statement 1: “A/though the laboratory’s submitted protocol 
indicates that an external positive OC material would be included in each run of patient 
Chlamydia trachomatis/Neisseria gonorrhoeae (CT /NG) testing, the laboratory provided no 
documentation of the positive OC material used, how the statistical parameters of the positive 
OC material would be determined, how the positive QC test result will be documented, and 
whether laboratory staff has been trained on this new protocol.” 


Theranos Response to Statement 1 — D5449: The laboratory has revised the CT/NG SOP 
to include information about the positive QC materials used, how the statistical parameters of 
the QC materials would be determined, and how QC test results will be documented. (Ex. 
BB, Tab 16 § 9.1.5). The relevant laboratory personnel have been trained on this protocol. 
(Ex. BB, Tabs 16B-16C). 


D5449 — March 18 Letter Statement 2: “Zo ensure the deficient practice does not recur, the 
laboratory indicated that quarterly audits will be performed and suggested that the audits results 
would be reviewed within the laboratory’s OMPI Program. However, the laboratory did not 
establish the procedure by which these quarterly audits are to be conducted. In the submission, 
the laboratory indicates that a ‘tracer audit may [emphasis added [in the letter]] be used, but 
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did not provided [sic] a protocol for a ‘tracer audit,’ the means by which a ‘tracer audit’ would 
be documented, and whether the results of a ‘tracer audit’ would be the information reviewed by 
the OMPI Program.” 


Theranos Response to Statement 2 — D5449: The laboratory has revised its audit 
procedures to establish the procedures by which audits are to be conducted. (Ex. BB, Tab 8). 
Training on these procedures has occurred. (Ex. BB, Tabs 8B-C). The revised procedures 
establish that tracer audits must occur monthly; provide the protocol for a tracer audit; and 
require that tracer audits be documented using the tracer audit form, CL FRM 00046 F2. 

(Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a quarterly audit to 
cover the quality management systems; provide the protocol for quarterly audits; and require 
that quarterly audits are documented using the audit record form, CL QOP 00046 F1. (Ex. 
BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear that “[b]oth 
tracer and quarterly audits are reviewed, analyzed and presented based on findings and 
analysis” during the QMPI Program meetings. (Ex. BB Tab 7§ 7.5). Training on the revised 
QMPI procedures has occurred. (Ex. BB, Tab7B-C). 


D5469 


Finding #1 


D5469 Finding #1 — March 18 Letter Statement 1: “‘A/though the laboratory’s submitted 
protocol indicates that the stated values of new commercially assayed CBC OC materials were 
to be verified through parallel testing against OC materials in use, the laboratory provided no 
documentation indicating that this protocol had been effectuated, no information as to how the 
results of the parallel testing will be documented, and no information as to whether laboratory 
staff has been trained on this new protocol.” 


Theranos Response to Statement 2 — D5469 Finding #1: D5469 Finding #1 relates to the 
laboratory’s “two Drew 3 instruments.” The laboratory’s two Drew-3 instruments were 
retired before the survey began on September 22, 2015. Since the survey, there has not yet 
been a need to perform parallel testing against QC materials in use because the laboratory has 
not approached a QC lot change for the tests that are in use. To ensure that there will not be 
an issue with documentation of such parallel testing in the future, the laboratory has further 
revised its procedures as to how the results of the parallel testing against QC materials in use 
must be documented. (Ex. BB, Tab 14 § 9). The laboratory staff has been trained on this 
documentation procedure. (Ex. BB, Tabs 14B-C). 


To ensure that there is not any confusion, we note that the Advia 21201 CBC SOP is inactive 
(and is not in use) because the laboratory stopped using those instruments on November 17, 
2015, and has not used them since then. 
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D5469 Finding #1 — March 18 Letter Statement 2: “Zo ensure the deficient practice does not 
recur, the laboratory indicated that quarterly audits will be performed and suggested that the 
audits results would be reviewed within the laboratory's OMPI Program. However, the 
laboratory did not establish the procedure by which these quarterly audits are to be conducted. 
In its submission, the laboratory indicates that a “tracer audit may [emphasis added] be used,” 
but did not provided a protocol for a “tracer audit,” the means by which a “tracer audit” would 
be documented, and whether the results of a “tracer audit” would be the information reviewed 
by the OMPI Program.” 


Theranos Response to Statement 2 — D5469 Finding #1: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 F1. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]oth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB Tab 7§ 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tab7B-C). 


Finding #2 


D5469 Finding #2 — March 18 Letter Statement 1: “A/though the laboratory’s submitted 
protocol indicates that the stated values of new commercially assayed chemistry OC materials 
were to be verified through parallel testing against OC materials in use, the laboratory provided 
no documentation indicating that this protocol had been effectuated, no information as to how 
the results of the parallel testing will be documented, and no information as to whether 
laboratory staff has been trained on this new protocol.” 


Theranos Response to Statement 1 — D5469 Finding #2: D5469 Finding #2 relates to the 
chemistry assays on the Siemens Advia 1800 and Siemens Advia XPT instruments. The 
laboratory stopped using the Siemens Advia 1800 instruments before the survey began on 
September 22, 2015, and has not used them since then. The laboratory stopped using the 
Siemens Advia XPT instrument on November 17, 2015, and has not used it since then. Since 
the survey, there has not yet been a need to perform parallel testing against QC materials in 
use because the laboratory has not approached a QC lot change for the tests that are in use. 
To ensure that there will not be an issue with documentation of such parallel testing in the 
future, the laboratory has further revised its procedures as to how the results of the parallel 
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testing against QC materials in use must be documented. (Ex. BB, Tab 14). The current 
procedures include worksheets for how to document parallel testing. (Ex. BB, Tab 14 F1-F4). 
The laboratory staff has been trained on this documentation procedure. (Ex. BB, Tabs 14B- 
14C). 


D5469 Finding #2 — March 18 Letter Statement 2: “Zo ensure the deficient practice does not 
recur, the laboratory indicated that quarterly audits will be performed and suggested that the 
audits results would be reviewed within the laboratory’s OMPI Program. However, the 
laboratory did not establish the procedure by which these quarterly audits are to be conducted. 
In its submission, the laboratory indicates that a “tracer audit may [emphasis added] be used,” 
but did not provided a protocol for a “tracer audit,” the means by which a “tracer audit” would 
be documented, and whether the results of a “tracer audit” would be the information reviewed 
by the OMPI Program.” 


Theranos Response to Statement 2 — D5469 Finding #1: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 Fl. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]oth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB Tab 7§ 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tab7B-C). 


D3477 


D5477 — March 18 Letter Statement 1: “Zhe laboratory submitted no written procedure for the 
bacteriology media QC protocol, and no information as to whether laboratory staff has been 
trained on this new protocol.” 


Theranos Response to Statement 1 — D5477: The laboratory has enclosed with this letter 
its written procedure for the bacteriology media QC protocol and documentation showing 
that the relevant personnel have been trained on it. (Ex. BB, Tabs 4-4C). 


D5477 — March 18 Letter Statement 2: “/n kx. L, Jab 8, the submission included a completed 
form titled ‘Blood Agar 5% Quality Control Log sheet.’ We note that the entry for the media 
received on 1/18/16 indicates that OC testing of this bacteriology media was completed on 
1/14/16, which is before the date the media was received by the laboratory. Yet, as documented 
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on the form, the completed form was reviewed on 2/11/16 without any indication this entry had 
been reviewed. We question the accuracy of this entry as well as the effectiveness of the 
laboratory’s oversight mechanism. ” 


Theranos Response to Statement 2 — D5477: CMS appears to have had difficulty reading 
the handwriting on the Blood Agar 5% Quality Control Log Sheet, confusing a handwritten 
“3” with an “8”. The “Blood Agar 5% Quality Control Log Sheet” states that the media was 
received on 1/13/16. ((Ex. HH, Tabs SA-5B). Therefore, there is no reason to question that 
accuracy of this entry because the log shows that the media was received the day before QC 
testing of this bacteriology media was completed. For the same reason, the February 11, 
2016 review of the completed form was performed correctly, and there is no reason to 
question the effectiveness of the laboratory’s oversight mechanism. 


D5469 Finding #2 — March 18 Letter Statement 3: “Zo ensure the deficient practice does not 
recur, the laboratory indicated that quarterly audits will be performed and suggested that the 
audits results would be reviewed within the laboratory’s OMPI Program. However, the 
laboratory did not establish the procedure by which these quarterly audits are to be conducted. 
In its submission, the laboratory indicates that a “tracer audit may [emphasis added] be used,” 
but did not provided a protocol for a “tracer audit,” the means by which a “tracer audit” would 
be documented, and whether the results of a “tracer audit” would be the information reviewed 
by the QMPI Program.” 


Theranos Response to Statement 3 — D5469 Finding #2: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 Fl. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]oth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB Tab 7§ 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tab7B-C). 


D5481 
Finding #1 


D5481 Finding #1 — March 18 Letter Statement 1: “Zhe submission references ‘Ex. I, Tabs 2- 
6.’ We located these tabs, but found no documentation in Tabs 2, 5, and 6.” 
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Theranos Response to Statement 1 — D5481 Finding #2: There was an inadvertent 
copying error for Exhibit I to the initial submission. The laboratory has enclosed the 
applicable documents with this letter, as well as additional documents related to the 
laboratory’s broader analysis of the PT/INR test over the entire time it was run on this 
Siemens BCS XP instrument. (Ex. GG). 


D5481 Finding #1 — March 18 Letter Statement 2: “A/though the laboratory’s submitted 
protocol requires that OC values be acceptable prior to reporting patient results, the submission 
states: ‘Theranos reviewed all quality control (QC) data for PTINR [Prothombin 
Time/International Normalized Ratio] for the time period that this lot of Dade Innovin was in 
use.’ The laboratory provided no documentation of this review other than stating it was 
performed. 


Theranos Response to Statement 2 — D5481 Finding #1: The laboratory has revised its 
patient assessment analysis for PT/INR, which is enclosed with this letter at Ex. AA, Tab 7. 
The revised assessment references supporting documentation identifying the time period that 
this lot of Dade Innovin was in use, as well as the QC data for that period. (Ex. AA, Tab 7; 
Ex. GG). That supporting documentation is also enclosed with this letter. (Ex. GG). 


D5481 Finding #1 — March 18 Letter Statement 3: We also found no documentation to 
indicate that the revised standard operating procedures (SOPS) have been effectuated. That is, 
we found no documentation of PTINR QC failure investigations and corrective actions taken 
based on the revised SOPS.” 


Theranos Response to Statement 3 — D5481 Finding #1: There appears to have been a 
miscommunication because the laboratory stopped running tests on the Siemens BCS XP, 
including PT/INR, on September 17, 2015, and has not run any tests on the instrument since 
then. Therefore, there have been no new QC failures for PT/INR to investigate or address 
through corrective action. The revised SOPs were implemented only after CMS identified 
the QC failures for PT/INR. The “review” discussed in the submission was simply intended 
to refer to the analysis conducted to address the deficiency identified by CMS. The issue 
addressed by the “review” — i.e., failing to document corrective action for QC fails for 
PT/INR — is the same issue identified by CMS in its deficiency, and thus did not constitute a 
new issue or occurrence. 


D5481 Finding #1 — March 18 Letter Statement 4: “/n Ex. /, the laboratory provided lists of 
patient specimen accession numbers for which the laboratory intended to issue corrected test 
reports. The laboratory provided no documentation to indicate corrected reports were issued. 
We also note that the Patient Impact Assessment (Ex. I, Tab 1) states that “remedial action was 
taken on 9/25/15,” but “corrected reports were issued beginning on 11/10/15 and completed on 
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11/12/15.” There was no explanation as to why there was such a long period of time between the 
remedial action and issuing corrected reports. 


Theranos Response to Statement 4 — D5481 Finding #1: All of these corrected reports for 
PT/INR, along with documentation confirming that they were sent, are enclosed with this 
letter. (Ex. GG, Tab 7). On September 22 or 23, CMS identified a number of issues related 
to PT/INR tests during the period April to September 2015. The laboratory undertook a 
thorough investigation. Given the number of issues to review, the investigation took time to 
complete properly. Timeliness for proper completion of the investigation and associated 
corrective actions were also affected by the absence of a full-time, on-site laboratory director. 
The laboratory subsequently hired its new lab director, Dr. Kingshuk Das, who is highly 
qualified and has extensive clinical laboratory experience. As of March 14, 2016, Dr. Das 
joined Theranos on-site full-time at the Newark, California laboratory. Additionally, the 
laboratory has hired Donald Tschirhart, M.D to be a full-time co-director (under California 
regulations) and a full-time clinical consultant. Dr. Tschirhart has over fifteen years of 
experience as a clinical laboratory director, and is a valuable part of the laboratory’s new 
leadership and oversight team. 


D5481 Finding #1 — March 18 Letter Statement 5: “70 ensure the deficient practice does not 
recur, the laboratory indicated that quarterly audits will be performed and suggested that the 
audits results would be reviewed within the laboratory’s OMPI Program. However, the 
laboratory did not establish the procedure by which these quarterly audits are to be conducted. 
In its submission, the laboratory indicates that a “tracer audit may [emphasis added] be used,” 
but did not provided a protocol for a “tracer audit,” the means by which a “tracer audit” would 
be documented, and whether the results of a “tracer audit” would be the information reviewed 
by the QMPI Program.” 


Theranos Response to Statement 5 — D5481 Finding #1: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 Fl. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]oth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB Tab 7§ 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tab7B-C). 
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Finding #2 


D5481 Finding #2 — March 18 Letter Statement 1: “A/though the laboratory submitted a OC 
protocol (Ex. A, Tab 9), the laboratory’s submission did not include an updated protocol for OC 
specific to the Theranos Proprietary Devices.” 


Theranos Response to Statement 1 — D5481 Finding #2: This aspect of the review appears 
to be based on a miscommunication because Theranos retired the TPS 3.5 in early-August 
2015. 


D5481 Finding #2 — March 18 Letter Statement 2: “7he reference protocol states at §§ 8.1.9- 
8.1.10: 


‘No instrument or test method can be used for the purpose of performing a test to report 
a result to a patient prior to having passed all required OC procedures at least once for 
the day on which the test is to be performed. It is the responsibility of all testing 
personnel to ensure this does not occur. Testing personnel must immediately take the 
appropriate action for all failed OC runs, refer to section 8.4.’ 


Tn addition, the laboratory's submission states: 


‘To perform this analysis, systematic error was investigated using quality control (QC) 
data and patient distribution over time. The OC data was reviewed in Levey-Jennings 
plots and evaluated for bias trends. The patient data was summarized by plotting monthly 
means, monthly medians and monthly means (“average of normal” or AON) calculated 
Srom central 95% of values. In addition, random errors were evaluated by analysis of OC 
data. Namely, OC data was reviewed to identify >2SD failures and period of elevated 
CV. Namely, batches of 10 consecutive OC data points were analyzed and the CV was 
calculated for each OC level during this period. If the CV was >30%, then all patient 
results run during that time period will be voided. (Ex. , Tab 1).’ 


The laboratory’s response does not include an evaluation of unacceptable QC results or patients 
affected or potentially affected by the cited OC failures. It is not clear how evaluating aggregate 
patient (i.e., monthly means and medians) and QC (i.e., bias trends, >2SD, CV) data addressed 
specific days that the OC was unacceptable and patient results were reported. 


The summary (Ex. E, Tab 1) included analytes tested on the Theranos Proprietary Device. We 
note that general statements such as, but not limited to, were included: 


- ‘Monthly patient distribution variance were noted, but they could not be further 
substantiated by OC trends’ 
- ‘Patient results following a OC failure and during periods of high CV are being voided’ 
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- ‘Patient distribution and QC trends did not reveal significant trends. ’ 
- ‘Potential biases observed in patient distributions were found to be insignificant relative 
to the normal reference intervals’ 


The laboratory’s submission did not provide any explanation or documentation to support these 
statements.” 


Theranos Response to Statement 2 — D5481 Finding #2: This deficiency concerns the 
TPS 3.5, which was one of the Theranos proprietary technologies in use in 2014 and 
2015. The laboratory began phasing out the TPS 3.5 in December 2014, and it was fully 
retired in early-August 2015. 


The laboratory values CMS’s feedback, and takes it very seriously. The laboratory has 
conducted a thorough re-review of QC data for each assay run on the TPS 3.5 from January 
1, 2014 until the TPS 3.5 was fully retired in early-August 2015. As explained further in the 
attached patient assessment analysis, the laboratory is not satisfied with its old quality 
assessment program’s ability to effectively flag and promptly address QC imprecision, QC 
failures, and QC trends on the TPS 3.5. (Ex. AA, Tab 3). The laboratory has new leadership 
and a new QMPI Program with robust quality monitoring and program improvement 
procedures. Based on its re-review of QC data for the TPS 3.5, the laboratory has 
determined that its prior QA program failed to satisfactorily address these types of QC issues 
for assays run on the TPS 3.5 in 2014 and 2015. As corrective action, the laboratory has, out 
of an extreme abundance of caution, voided results reported for assays run on the TPS 3.5 in 
2014 and 2015. (Ex. AA, Tab 3). Many corrected reports have been transmitted, and the 
remainder are being transmitted. (Ex. KK). Transmission will be complete by March 31, 
2016. The remainder of the transmittals and confirmations of receipt will be provided to 
CMS under separate cover. 


D5481 Finding #2 — March 18 Letter Statement 3: “Zhe submission (Ex. E) includes a list of 
patient specimen accession numbers for which the laboratory “identified reports,” but does not 
specify or include documentation to indicate whether corrected reports were generated and 
issued.” 


Theranos Response to Statement 3 — D5481 Finding #2: As indicated above, the 
laboratory has revised and broadened its analysis of this deficiency. Based on that analysis, 
which is described above and in the attached documentation, the laboratory has, out of an 
extreme abundance of caution, voided all results reported for assays run on the TPS 3.5 in 
2014 and 2015. (Ex. AA, Tab 3). Many corrected reports have been transmitted, and the 
remainder are being transmitted. (Ex. KK). Transmission will be complete by March 31, 
2016. The remainder of the transmittals and confirmations of receipt will be provided to 
CMS under separate cover. 


D5775 
Finding #1 
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D5775 Finding #1 — March 18 Letter Statement 1: “Ji the submitted protocol, the laboratory 
states: ‘To ensure all manual differentials performed are comparable, ... manual differentials 
will be compared annually amongst the competent staff...’ The laboratory provided no written 
procedure as to how “manual differentials will be compared amually.’ In addition, the 
regulation requires the laboratory to conduct such evaluations twice a year, not annually.” 


Theranos Response to Statement 1 — D5775: The laboratory has revised the Cellavision 
SOP to include procedures for how manual differential will be compared and to require that 
the laboratory perform such evaluations twice a year. (Ex. BB, Tab 15 § 9.1.4.2). Please 
note, however, that this SOP is inactive because the laboratory stopped using the Cellavision 
on November 17, 2015, and has not used it since then. 


D5775 Finding #1 — March 18 Letter Statement 2: “/o ensure the deficient practice does not 
recur, the laboratory indicated that quarterly audits will be performed and suggested that the 
audits results would be reviewed within the laboratory’s QMPI Program. However, the 
laboratory did not establish the procedure by which these quarterly audits are to be conducted. 
Tn its submission, the laboratory indicates that a “tracer audit may [emphasis added] be used,” 
but did not provided a protocol for a “tracer audit,” the means by which a “tracer audit” would 
be documented, and whether the results of a “tracer audit” would be the information reviewed 
by the OMPI Program.” 


Theranos Response to Statement 2 — D5775 Finding #1: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 Fl. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]oth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB Tab 7§ 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tab7B-C). 


Finding #2 


D5775 Finding #2 — March 18 Letter Statement 1: “A/though the laboratory submitted a 
method instrument comparison protocol (Ex. A, Tab 28), we were unable to determine if the 
protocol was effectuated. In its submission, the laboratory states: ‘The lab’s technical 
supervisors and the quality system director will be responsible for ensuring that these 
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procedures are followed.’ However, the laboratory’s submission did not include any 
documentation to indicate that the technical supervisors or the quality systems director had been 
trained on the method/instrumentation comparison protocols.” 


Theranos Response to Statement 1 — D5775 Finding #2: This deficiency concerns method 
comparison for the TPS 3.5. As indicated above, the TPS 3.5 was fully retired in early- 
August 2015. Before the survey began, the laboratory had stopped running the same test 
using different methodologies or instruments, and the laboratory has not run the same test 
using different methodologies or instruments since then. As a result, the laboratory has had 
no reason to use the method verification protocol that it included with the submission. 
Although the laboratory does not have a need to use the revised method comparison protocol 
at this time, it has trained its technical supervisors and quality director on it. (Ex. BB, Tabs 
12-12C). 


D5775 Finding #2 — March 18 Letter Statement 2: “Zhe laboratory's submission also 
includes the following statement: 


Because 1PS’s [| Theranos Proprietary Devices] were factory calibrated by Theranos, 
the laboratory did not perform additional method correlation studies. As part of this 
investigation, Theranos examined the correlation of different TPS to evaluate their 
concordance. Namely, for SHGB, TT3, Vitamin B12 and Vitamin D, patient 
distributions were compared across representative TPS’s in use during the same 
period (Exhibit E Tab 12-A; Tab 24-A, Tab 33-A). These data showed similar patient 
distributions across the different devices. In addition, comparison of assay results 
across different assay levels and different TPS's demonstrates that the instruments 
were well correlated to each other (Exhibit F., Tab 12-A; Tab 24-A, Tab 33-A). 
Namely, the median result for each TPS was within the total allowable error thereby 
demonstrating that the same assay reference ranges were appropriate for TPS. 


The laboratory's response indicates that not all TPS would be included in the correlation studies 
rather ‘across representative TPS in use during the same period.’ The laboratory submitted 
correlation studies for Sex Hormone Binding Globulin (SHBG) and Vitamin D (VitD) only (Ex E, 
Tabs 12-A, 24-A and 33-A, respectively). However, these studies did not include a date on either 
the summary or raw data, so we were unable to determine when the correlation studies 
occurred, The submission for SHBG also included information on one document with box plots 
that included three devices and another document that included box plots of six devices, so we 
were unable to determine what devices were actually used for SHBG testing. 


The laboratory’s submission offered no response to correlation studies for Vitamin B12 and no 
explanation for the disparity between the devices used for OC and the devices used for patient 
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testing. We also noted that there was no explanation as to where the total allowable error (TAE) 
of 24% on the raw data documentation came from or if any TAL was calculated for the submitted 
data. The laboratory used the median result for each proprietary device to determine if TAE was 
acceptable, but did not provide an explanation why the median results were used.” 


Theranos Response to Statement 2 — D5775 Finding #2: This portion of CMS’s March 18 
letter concerns aspects of the patient assessment analysis for tests run on the TPS 3.5. The 
laboratory began phasing out the TPS 3.5 in December 2014, and it was fully retired in 
early-August 2015. 


As reflected in its submission, the laboratory believes oversight, implementation and 
structure of its prior method comparison procedures were not satisfactory, which is why it 
has developed the improved method comparison procedures that were included with the 
submission. The laboratory has further revised those procedures in response to other 
feedback contained in CMS’s March 18 letter. (Ex. BB, Tab 12). Training on these 
procedures has occurred. (Ex. BB, Tabs 12B-12C). 


The laboratory values CMS’s feedback, and takes it very seriously. The laboratory has 
conducted a thorough re-review of QC data for each assay run on the TPS 3.5 from January 
1, 2014 until the TPS 3.5 was fully retired in early-August 2015. As explained further in the 
attached patient assessment analysis, the laboratory is not satisfied with its old quality 
assessment program’s ability to effectively flag and promptly address QC imprecision, QC 
failures, and QC trends on the TPS 3.5. (Ex. AA, Tab 3). The laboratory has new leadership 
and a new QMPI Program with robust quality monitoring and program improvement 
procedures. Based on its re-review of QC data for the TPS 3.5, the laboratory has 
determined that its prior QA program failed to satisfactorily address these types of QC issues 
for assays run on the TPS 3.5 in 2014 and 2015. As corrective action, the laboratory has, out 
of an extreme abundance of caution, voided results reported for assays run on the TPS 3.5 in 
2014 and 2015. (Ex. AA, Tab 3). Many corrected reports have been transmitted, and the 
remainder are being transmitted. (Ex. KK). Transmission will be complete by March 31, 
2016. The remainder of the transmittals and confirmations of receipt will be provided to 
CMS under separate cover. 


D5775 Finding #2 — March 18 Letter Statement 3: “Zo ensure the deficient practice does not 
recur, the laboratory indicated that quarterly audits will be performed and suggested that the 
audits results would be reviewed within the laboratory’s OMPI Program. However, the 
laboratory did not establish the procedure by which these quarterly audits are to be conducted. 
In its submission, the laboratory indicates that a “tracer audit may [emphasis added] be used,” 
but did not provided a protocol for a “tracer audit,” the means by which a “tracer audit” would 
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be documented, and whether the results of a “tracer audit” would be the information reviewed 
by the OMPI Program.” 


Theranos Response to Statement 2 — D5775 Finding #1: audits are to be conducted. (Ex. 
BB, Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 00046 
F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a quarterly audit 
to cover the quality management systems; provide the protocol for quarterly audits; and require 
that quarterly audits are documented using the audit record form, CL QOP 00046 F1. (Ex. BB, 
Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear that “[b]oth tracer and 
quarterly audits are reviewed, analyzed and presented based on findings and analysis” during 
the QMPI Program meetings. (Ex. BB Tab 7§ 7.5). Training on the revised QMPI procedures 
has occurred. (Ex. BB, Tab7B-C). 


D5779 


D5779 — March 18 Letter Statement 1: “/n x. /, the laboratory provided lists of patient 
specimen accession numbers for which the laboratory intended to issue corrected test reports. 
The laboratory provided no documentation to indicate corrected reports were issued.” 


Theranos Response to Statement 1 — D5779: Many corrected reports have been 
transmitted, and the remainder are being transmitted. (Ex. KK). Transmission will be 
complete by March 31, 2016. The remainder of the transmittals and confirmations of receipt 
will be provided to CMS under separate cover. 


D5779 — March 18 Letter Statement 2: “Zo ensure the deficient practice does not recur, the 
laboratory indicated that quarterly audits will be performed and suggested that the audits results 
would be reviewed within the laboratory’s QMPI Program. However, the laboratory did not 
establish the procedure by which these quarterly audits are to be conducted. In its submission, 
the laboratory indicates that a “tracer audit may [emphasis added] be used,” but did not 
provided a protocol for a “tracer audit,” the means by which a “tracer audit” would be 
documented, and whether the results of a “tracer audit” would be the information reviewed by 
the OMPI Program.” 


Theranos Response to Statement 2 — D5779: The laboratory has revised its audit 
procedures to establish the procedures by which audits are to be conducted. (Ex. BB, Tab 8). 
Training on these procedures has occurred. (Ex. BB, Tabs 8B-C). The revised procedures 
establish that tracer audits must occur monthly; provide the protocol for a tracer audit; and 
require that tracer audits be documented using the tracer audit form, CL FRM 00046 F2. 

(Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a quarterly audit to 
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cover the quality management systems; provide the protocol for quarterly audits; and require 
that quarterly audits are documented using the audit record form, CL QOP 00046 F1. (Ex. 
BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear that “[b]oth 
tracer and quarterly audits are reviewed, analyzed and presented based on findings and 
analysis” during the QMPI Program meetings. (Ex. BB Tab 7§ 7.5). Training on the revised 
QMPI procedures has occurred. (Ex. BB, Tab7B-C). 


D5787 


D5787 — March 18 Letter Statement 1: “Zhe laboratory submitted no written procedure for 
the documentation of the lot of bacteriology media used to test any given patient specimen, and 
no information as to whether laboratory staff has been trained on this new protocol.” 


Theranos Response to Statement 1 — D5787: The laboratory has enclosed with this letter 
its written procedure for the documentation of bacteriology media QC protocol. (Ex. BB, 
Tab 4 § 8.2). The relevant laboratory personnel have been trained on that procedure, as 
reflected in the documentation enclosed at Ex. BB, Tabs 4B-4C. 


D5787 — March 18 Letter Statement 2: “Jn its submission, the laboratory references Lx. L, 
Tab 6 which states: “When [media quality control] testing is completed the results are logged 
into an excel sheet with lot number and expiration date for each patient (see attached report).” 
We find no “attached report” in the submission and, therefore, no documentation indicating the 
laboratory’s corrective action has been effectuated.” 


Theranos Response to Statement 2 — D5787: The laboratory has effectuated its written 
bacteriology media QC protocol by logging each patient result into a separate excel 
worksheet that includes the lot number and expiration date for the bacteriology media lot 
used for that patient test. We have enclosed examples of these excel spreadsheets, which 
show that the laboratory’s corrective action has been effectuated. (Ex. HH, Tab 6). 


D5787 — March 18 Letter Statement 3: “Zo ensure the deficient practice does not recur, the 
laboratory indicated that quarterly audits will be performed and suggested that the audits results 
would be reviewed within the laboratory’s OMPI Program. However, the laboratory did not 
establish the procedure by which these quarterly audits are to be conducted. In its submission, 
the laboratory indicates that a “tracer audit may [emphasis added] be used,” but did not 
provided a protocol for a “tracer audit,” the means by which a “tracer audit” would be 
documented, and whether the results of a “tracer audit” would be the information reviewed by 
the OMPI Program.” 


Theranos Response to Statement 3 — D5787: The laboratory has revised its audit 
procedures to establish the procedures by which audits are to be conducted. (Ex. BB, Tab 8). 
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Training on these procedures has occurred. (Ex. BB, Tabs 8B-C). The revised procedures 
establish that tracer audits must occur monthly; provide the protocol for a tracer audit; and 
require that tracer audits be documented using the tracer audit form, CL FRM 00046 F2. 

(Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a quarterly audit to 
cover the quality management systems; provide the protocol for quarterly audits; and require 
that quarterly audits are documented using the audit record form, CL QOP 00046 F1. (Ex. 
BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear that “[b]oth 
tracer and quarterly audits are reviewed, analyzed and presented based on findings and 
analysis” during the QMPI Program meetings. (Ex. BB Tab 7§ 7.5). Training on the revised 
QMPI procedures has occurred. (Ex. BB, Tab7B-C). 


D5791 


Finding #1 


D5791 Finding #1 — March 18 Letter Statement 1: “Zhe /aboratory submitted a protocol for 
the SensoScientific Monitoring in Ex. A, Tab 24. The response does not specifically address the 
days that the freezers did not meet acceptable temperatures as documented in the SensoScientific 
Monitoring Audit Node Log. We could not determine if the new procedure has been effectuated 
as no documentation was submitted to show that the unacceptable freezer temperatures were 
addressed. 


Theranos Response to Statement 1 — D5791 Finding #1: The laboratory has effectuated its 
revised procedures for the SensoScientific Monitoring that were included with our initial 
submission. Two examples of the current system monitoring are enclosed with this letter. 
(Ex. Il, Tabs 1A-1B). First, on March 18, 2016, at about 11:00 a.m., the laboratory’s Wi-Fi 
became temporarily inoperable. All monitors went into failure mode and a down-time paper 
recording was initiated. The Wi-Fi was brought back up and the system was restored by 6:00 
p.m. Data was reclaimed from the monitors for the period in question, and this failure was 
documented in the enclosed Alarm History and Corrective Action (Ex. II, Tab 1A) and the 
enclosed Occurrence Report Form. (Ex. I, Tab 1B). Second, on March 19, 2016, freezer 
7063 failed a temperature check. The low magnitude and brevity of the failure was below 
the alarm threshold. (Ex. Il, Tab 4). The failure was detected and evaluated on a weekly 
review. (Ex. II, Tab 4). As it had corrected itself less than 30 minutes after it occurred, there 
was no need for further action. (Ex. Il, Tab 4). This demonstrates that the procedure is being 
followed and temperatures are being adequately monitored. In addition, as stated in above 
regarding D5413, the laboratory has also posted signs on the freezers pursuant to the new 
procedures, as reflected in the attached documentation. (Ex. II, Tab 2). 
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D5791 Finding #1 — March 18 Letter Statement 2: /n addition, the laboratory states that 
training has occurred, but it was not clear who should be trained as ‘laboratory supervisor or 
designee’ was not defined.” 


Theranos Response to Statement 2 — D5791 Finding #1: The laboratory has revised this 
procedure to clarify that “[i]t is the daily responsibility of the laboratory supervisor to 
monitor and record the temperatures of refrigerators and freezers and the temperature and 
humidity of each lab room and ambient storage space.” (Ex. BB, Tab 13 § 4.5). The revised 
procedure is enclosed at Ex. BB, Tab. 13. 


D5791 Finding #1 — March 18 Letter Statement 3: “7raining records provided in Ex. A, Tab 
30 included only training documents for one general supervisor.” 


Theranos Response to Statement 3 — D5791 Finding #1: The laboratory has enclosed with 
this letter a complete set of training records on this SOP, which show that the relevant 
laboratory personnel have been trained. (Ex. BB, Tabs 13B-13C). 


D5791 Finding #1 — March 18 Letter Statement 4: “Several freezers were identified as ‘not 
used in patient testing’ (see Ex. N, Tab A); however, based on interviews during the onsite visit, 
these freezers were identified as being used for CLIA activities. The submission did not include 
a response or data related to the freezers identified as ‘not used in patient testing.’” 


Theranos Response to Statement 4 — D5791 Finding #1: D5791 #1 identifies the same six 
-20 C freezers and four -80 freezers as D5413. In the laboratory’s submission, we explain 
that five of those freezers did not contain materials that would be used in the future for 
clinical patient testing. Rather, they only contained material that might be used in the future 
for research and development. 


Based on CMS’s March 18 letter, it appears that there was a miscommunication when the 
CMS surveyors were onsite. D5413, Paragraph f, states that CMS had a discussion about 
these freezers with the “Director of Assay Systems and technical supervisor at 11/19/15 at 
approximately 11 am.” The Director of Assay Systems did not have any regular involvement 
with those freezers at that time and did not have knowledge of their contents. The “technical 
supervisor” in attendance on November 19, 2015 joined the company on October 26, 2015 
(Ex. HH, Tab 8); he also did not have regular involvement with those freezers at the time and 
did not have knowledge of their contents. To the extent CMS understood either of these 
individuals to be confirming the contents of these freezers or whether these freezers were 
used in clinical patient testing, there was a miscommunication. 


The laboratory has looked into this issue further, and has reconfirmed that the statements in 
the laboratory’s submission are accurate. As reflected in the enclosed documentation: 


e On November 19, 2015, there were four -20 C freezers in the microbiology or BUGS 
laboratory. 7061 BUGS -20 C Freezer 4 did not contain any materials for future use 
in clinical patient testing. (Ex. II, Tab 3B). 
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e On November 19, 2015, there was one -20 C freezer in the capillary laboratory. 
Clinical patient testing in the Normandy laboratory stopped on September 20, 2015, 
and there has not been any clinical patient testing in the Normandy laboratory since 
then. The 1 Normandy -20 C freezer 7066 did not contain any materials for future 
use in patient testing. (Ex. II, Tab 3C). 


e On November 19, 2015, there were four -80 C freezers. 7098 -80 C Freezer 1 Nuair, 
7111 -80 C Freezer 2 Thermo, and 7113 -80 2 CLIA Lab did not contain any 
materials for future use in clinical patient testing. While the last of those three 
freezers was named 7113 -80 2 CLIA Lab, it was only used to store old specimens 
that were not for future use in clinical patient testing. (Ex. II, Tab 3A) 


D5791 Finding #1 — March 18 Letter Statement 5: “Zhe /aboratory submitted an article for 
the use of Mean Kinetic Temperatures (MKT) (Ex. N, Tab 1). This article relates to the handling, 
storage, and distribution of temperature sensitive pharmaceuticals, not clinical laboratory 
specimens, reagents, reference materials, etc. We are unclear as to how this article applies to 
this deficient practice, and note that the written procedure submitted by the laboratory at Ex. A, 
Tab 24 does not incorporate monitoring temperature by MKT.” 


Theranos Response to Statement 5 — D5791 Finding #1: Although the article is not critical 
to our analysis, it is relevant because it provides a specific rationale for understanding the 
chemical effects of brief increases (or decreases) in temperature on pharmaceuticals, and by 
extrapolation, on chemical controls. The conclusion of the article is that brief periods of 
temperature elevation have little effect on the rate of chemical degradation on the product being 
stored. While we do not recommend storing chemical control material in an unstable 
environment, the article gives reassurance that chemical control materials subjected to a short 
period of increased temperature by mistake are unlikely to be chemically degraded. 


D5791 Finding #1 — March 18 Letter Statement 6: ‘“/he laboratory also submitted an article 
related to the storage of bacterial samples for optimal viability (Ex. N, Tab 22). We note that the 
submitted written procedure relates to determining acceptable temperature viability of bacterial 
samples does not include any references to this article. We are unclear as to how this article 
applied to this deficient practice.” 


Theranos Response to Statement 6 — D5791 Finding #1: This article is cited in the 
laboratory’s revised patient assessment for this deficiency, which is enclosed with this letter. 
(Ex. AA, Tab 4). This article is relevant because it shows that bacterial samples will remain 
viable for 10 years at -80C and for 1-3 years at -20C. 


D5791 Finding #1 — March 18 Letter Statement 7: “Zhe /aboratory submitted no evidence to 
support the following conclusion for the Sanyo JP Lab 7059 -20 freezer: ‘[The laboratory] 
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determined that higher storage temperature had no effect on the calibrators or controls stored in 
this freezer. There is no patient impact from the change in temperature during the time period.’” 


Theranos Response to Statement 7 — D5791 Finding #1: The laboratory has revised its 
patient assessment analysis to provide more detail and documentation about its review of the 
effect this issue may have had on the Immulite controls stored in this freezer. (Ex. AA, Tab 
4). Based on that review, the laboratory has voided certain test results. (Ex. AA, Tab 4). 
Many corrected reports have been transmitted, and the remainder are being transmitted. (Ex. 
KK). Transmission will be complete by March 31, 2016. The remainder of the transmittals 
and confirmations of receipt will be provided to CMS under separate cover. 


D5791 Finding #1 — March 18 Letter Statement 8: /n addition, the laboratory stated for 
another freezer (1 BUGS 7120 -80 freezer) that the ‘MKT remained between -70 and -86 C for 
the entire date range.’ However, the laboratory indicated that “a small number of bacterial 
cultures ... labeled to be stored at -80C or colder” were stored in this freezer. It was unclear as 
to why it was appropriate to store ‘bacterial cultures ... labeled to be stored at -80C or colder’ to 
be stored at temperatures of -70C to -79C.” 


Theranos Response to Statement 8 — D5791 Finding #1: The temperature at which these 
bacterial cultures are stored is not determined by a manufacturer. While the laboratory 
recognizes that it should have followed its procedures for maintaining the temperature of this 
freezer at -80C or colder, storage of these bacterial cultures at temperatures of -70C to -79C 
did not have the potential to affect patients receiving tests in 2014 and 2015. Most 
importantly, these bacterial cultures can only be used in patient testing after there is growth, 
which occurs after the bacterial culture is removed from the freezer. If there is no growth 
after the bacterial culture is removed from the freezer, then the culture is not viable and 
cannot be used in patient testing. Therefore, the higher temperature of this freezer did not 
have any potential to affect patients, even if even it had caused a bacterial culture to be 
unviable. 


It is also worth noting that the temperature of -70C to -79C should not have had an impact on 
the viability of these bacterial cultures, which were put in these freezers after the laboratory 
moved to the Newark facility in October 2014. The laboratory labels bacterial cultures for 
storage at -80C or colder only because that allows for the longest period of viability, which is 
10 years. (Ex. II, Tab 26). However, bacterial cultures remain viable for 1-3 years when 
stored at -20C. (Ex. II, Tab 26). 


DS791 Finding #1 — March 18 Letter Statement 9: “Zo ensure the deficient practice does not 
recur, the laboratory indicated that quarterly audits will be performed and suggested that the 
audits results would be reviewed within the laboratory’s OMPI Program. However, the 
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laboratory did not establish the procedure by which these quarterly audits are to be conducted. 
In its submission, the laboratory indicates that a “tracer audit may [emphasis added] be used,” 
bul did not provided a protocol for a “tracer audit,” the means by which a “tracer audit” would 
be documented, and whether the results of a “tracer audit” would be the information reviewed 
by the QMPI Program.” 


Theranos Response to Statement 9 — D5791 Finding #1: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-8C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 F1. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]oth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB, Tab 7 § 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tabs 7B-7C). 


Finding #2 


D5791 Finding #2 — March 18 Letter Statement 1: “Zhe laboratory submitted a protocol for 
a Master Validation Plan (Ex. Ek, Tab 30B) which required the %CV of replicates to be not more 
than 20% (25% at the lower and upper limits of detection). The submitted protocol was different 
than the protocol presented to the surveyor at the time of the onsite survey which required the 
%CV of replicates to be not more than 15% (20% at the lower and upper limits of detection). 
The %CYV in the protocol presented to the surveyor at the time of the onsite survey matched the 
%CV of the predicate devices. It is unclear as to why the TPS, which is covered by the Master 
Validation Plan, can now have %CV’s for replicates greater than the predicate devices; no 
explanation was submitted. In addition, we note that the %CV’s (18.7% - 63.8%) included in the 
submission does not appear to have been evaluated using this new protocol nor was it addressed 
in the laboratory’s response provided at Ex. E, Tab 1.” 


Theranos Response to Statement 1 — D5791 Finding #2: This deficiency concerns the 
TPS 3.5. As noted above, the TPS 3.5 was fully retired in early-August 2015. 


With respect to these validation plans, there appears to have been a miscommunication. 
During the September 22-23, 2015 on-site visit, the laboratory understood that CMS had 
requested the validation plan for proprietary chemistry assays run on other Theranos 
technologies. The laboratory did not realize that CMS believed this plan was also the 
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validation plan that applied to ELISA assays run on the TPS 3.5. After reviewing the Form 
2567, we realized that there had been a miscommunication, which is why our submission 
included the ELISA Master Validation Plan. 


As detailed above, the laboratory values CMS’s feedback, and takes it very seriously. '* The 
laboratory has conducted a thorough re-review of QC data for each assay run on the TPS 3.5 
from January 1, 2014 until the TPS 3.5 was fully retired in early-August 2015. As 
explained further in the attached patient assessment analysis, the laboratory is not satisfied 
with its old quality assessment program’s ability to effectively flag and promptly address QC 
imprecision, QC failures, and QC trends on the TPS 3.5. (Ex. AA, Tab 3). The laboratory 
has new leadership and a new QMPI Program with robust quality monitoring and program 
improvement procedures. Based on its re-review of QC data for the TPS 3.5, the laboratory 
has determined that its prior QA program failed to satisfactorily address these types of QC 
issues for assays run on the TPS 3.5 in 2014 and 2015. As corrective action, the laboratory 
has, out of an extreme abundance of caution, voided results reported for assays run on the 
TPS 3.5 in 2014 and 2015. (Ex. AA, Tab 3). Many corrected reports have been transmitted, 
and the remainder are being transmitted. (Ex. KK). Transmission will be complete by March 
31, 2016. The remainder of the transmittals and confirmations of receipt will be provided to 
CMS under separate cover. 


D5791 Finding #2 — March 18 Letter Statement 2: “Zhe submission (Ex. E) includes a list of 
patient specimen accession numbers for which the laboratory “identified reports,” but does not 
specify or include documentation to indicate whether corrected reports were generated and 
issued.” 


Theranos Response to Statement 2 — D5791 Finding #2: As indicated above, the 
laboratory has revised and broadened its analysis of this deficiency and associated corrective 
actions. Based on that analysis, which is described above and in the attached documentation, 
the laboratory has, out of an extreme abundance of caution, voided all results reported for 
assays run on the TPS 3.5 in 2014 and 2015. (Ex. AA, Tab 3). Many corrected reports have 
been transmitted, and the remainder are being transmitted. (Ex. KK). Transmission will be 


'S In discussing establishment of precision for an LDT, the March 18 letter references a 
comparison of %CVs between an LDT and “predicate devices.” We note, however, that CLIA 
regulations do not contain any regulation requiring that there be any correlation or equivalence of 
%CV between LDTs and predicate devices. Rather, the regulations require only that the 
precision for the test must be established for the LDT itself. 42 C_F.R. § 493.1253(b)(2). It also 
is generally accepted that accuracy is the only required performance specification that typically 
requires a correlation study. 42 C.F.R. § 493.1253(b)(2). 
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complete by March 31, 2016. The remainder of the transmittals and confirmations of receipt 
will be provided to CMS under separate cover. 


Finding #3 


D5791 Finding #3 — March 18 Letter Statement 1: “Zhe laboratory's review of the OC found 
in Ex. FE, Tab 1 did not specify what period of OC the laboratory reviewed.” 


Theranos Response to Statement 1 — D5791 Finding #3: As indicated above, the 
laboratory’s analysis covers the entire period in which each assay run on the TPS 3.5 was in 
use in 2014 and 2015. (Ex. AA, Tab 3). 


D5791 Finding #3 —- March 18 Letter Statement 2: “Zhe quality assessment (QA) 
documentation related to OC data from the survey in July 2014, October 2014, and February 
through June 2015. The laboratory's investigation did not indicate that the review had identified 
the cited issues related to OC.” 


Theranos Response to Statement 2 — D5791 Finding #3: The revised analysis enclosed 
with this letter makes clear that the laboratory identified the cited issues related to QC. (Ex. 
AA, Tab 3). The laboratory’s review of QC in connection with the submission, as well as the 
re-review reflected in the enclosed revised analysis, included a review of imprecision, QC 
trends, and QC failures, including failures occurring where one of the two levels of control 
had a value of 2SD or greater. 


D5791 Finding #3 — March 18 Letter Statement 3: ‘70 ensure the deficient practice does not 
recur, the laboratory indicated that quarterly audits will be performed and suggested that the 
audits results would be reviewed within the laboratory’s OMPI Program. However, the 
laboratory did not establish the procedure by which these quarterly audits are to be conducted. 
In its submission, the laboratory indicates that a “tracer audit may [emphasis added] be used,” 
but did not provided a protocol for a “tracer audit,” the means by which a “tracer audit” would 
be documented, and whether the results of a “tracer audit” would be the information reviewed 
by the QMPI Program.” 


Theranos Response to Statement 3 — D5791 Finding #3: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-8C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 F1. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]oth tracer and quarterly audits are reviewed, analyzed and presented based on 
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findings and analysis” during the QMPI Program meetings. (Ex. BB, Tab 7 § 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tabs 7B-7C). 


Finding #4 


D5791 Finding #4 — March 18 Letter Statement 1: “‘A/though the laboratory's submitted 
protocol included information about quarterly audits and agenda items for the laboratory's 
QOMPI Program, the submission does not provide specific actions to take when problems are 
identified, specifically related to clotted specimens. The laboratory states in Ex. L, Tab 35 that 
“[p]atient specimens that did not meet the lab’s acceptance criteria were rejected ... “ However, 
the issue was not that the clotted specimens were rejected, rather that the laboratory’s QA 
program did not identify the high number of clotted specimens over a period of three quarters. 
The submission does not include any documentation showing that the issue was identified and 
corrected. We note that Ex. A, Tab 12, §7.2.1.6 indicates that the OMPI Program agenda 
included specimen rejection rate partitioned by rejection criteria, but does not require that any 
action be taken based on this agenda item. In addition, we note that in Ex. A, Tab 12, §7.1, the 
laboratory provided the frequency of the OMPI Program meetings, but does not indicate any 
action Should be taken in the event it is determined that the quality metrics were not met.” 


Theranos Response to Statement 1 — D5791 Finding #4; The laboratory has further 
revised its QMPI Program procedures to ensure that the QMPI Program identifies specimen 
rejection issues promptly. (Ex. BB, Tab 7 § 7.2.3.1). The procedure requires that when 
analyzing specimen rejection and redraws, the QMPI team will “[i]dentify any corrective 
action to take based on data such as retraining or a procedure change.” (Ex. BB, Tab 7 § 
72.3:16) 


D5791 Finding #4 — March 18 Letter Statement 2: “Zo ensure the deficient practice does not 
recur, the laboratory indicated that quarterly audits will be performed and suggested that the 
audits results would be reviewed within the laboratory's OMPI Program. However, the 
laboratory did not establish the procedure by which these quarterly audits are to be conducted. 
In its submission, the laboratory indicates that a “tracer audit may [emphasis added] be used,” 
but did not provided a protocol for a “tracer audit,” the means by which a “tracer audit” would 
be documented, and whether the results of a “tracer audit” would be the information reviewed 
by the QMPI Program.” 


Theranos Response to Statement 2 — D5791 Finding #4: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tab 8B-8C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
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quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 F1. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]oth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB, Tab 7 § 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tabs 7B-7C). 


D5793 


Finding #1 


The March 18 Letter states: “See our reviews of D5403, D5437, D5469, and D5779.” 
Accordingly, we refer you to our responses to those reviews. 


Finding #2 


D5793 Finding #2 — March 18 Letter Statement 1: /n the submission, the laboratory states: 


Based on comprehensive review, multiple examples of failed QC without the appropriate 
documentation with no investigation or corrective action was identified which is now 
addressed with revised SOPS and training. Additional competency is on-going to ensure 
CLS demonstrate the required proficiency with quality control and the appropriate 
investigation, corrective action and notification. 


We find no documentation to indicate that the ‘revised SOPS’ have been effectuated. That is, we 
find no documentation of cytometry QC failure investigations and corrective actions taken based 
on the ‘revised SOPS’ as discovered while conducting the ‘comprehensive review.’” 


Theranos Response to Statement 1 — D5793 Finding #2: There appears to have been a 
miscommunication here because the laboratory stopped running tests on the flow cytometer 
before the survey began on September 22, 2015, and has not run any tests on the instrument 
since then. Because the flow cytometer is not in use, and has not been in use since the survey 
began, there has not been any new QC failures that required investigation or corrective action 
under the laboratory’s new procedures. The “review” discussed in the submission was 
simply intended to refer to the analysis conducted to address the deficiency identified by 
CMS. The issue addressed by the “review” — i.e., failing to document corrective action for 
QC fails for the Fortessa and Canto — is the same issue identified by CMS in its deficiency, 
and thus did not constitute a new issue or occurrence. 


D5793 Finding #2 — March 18 Letter Statement 2: /n Ex. /, the laboratory provided lists of 
patient specimen accession numbers for which the laboratory intended to issue corrected test 


53 
TRADE SECRET/CONFIDENTIAL COMMERCIAL INFORMATION 
EXEMPT FROM DISCLOSURE UNDER FREEDOM OF INFORMATION ACT 


FOIA Confidential Treatment Requested by Theranos THER-0534752 
Fed. R. Crim. P. 6(e) material 


Case: 2281680226642) 00 Bodo meaiGeee, Ditechtty/Aw2I, Page 55 of 94 


thera Meo 1701 Page Mill Road P 650.838.9292 theranos.com 


Palo Alto, CA 94304 F 650.838.9165 


reports. The laboratory provided no documentation to indicate corrected reports were generated 
and issued. 


Theranos Response to Statement 2 — D5793 Finding #2: Many corrected reports have 
been transmitted, and the remainder are being transmitted. (Ex. KK). Transmission will be 
complete by March 31, 2016. The remainder of the transmittals and confirmations of receipt 
will be provided to CMS under separate cover. 


D5793 Finding #2 — March 18 Letter Statement 3: “7o ensure the deficient practice does not 
recur, the laboratory indicated that quarterly audits will be performed and suggested that the 
audits results would be reviewed within the laboratory’s QMPI Program. However, the 
laboratory did not establish the procedure by which these quarterly audits are to be conducted. 
In its submission, the laboratory indicates that a “tracer audit may [emphasis added] be used,” 
but did not provided a protocol for a “tracer audit,” the means by which a “tracer audit” would 
be documented, and whether the results of a “tracer audit” would be the information reviewed 
by the OMPI Program.” 


Theranos Response to Statement 3 — D5793 Finding #2: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-8C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 Fl. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]oth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB, Tab 7 § 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tabs 7B-7C). 


Finding #3 
D5793 Finding #3 — March 18 Letter Statement 1: “Jn the submission, the laboratory states: 


- ‘07/16/2015: Level 3 OC outside manufacturer ’s recommended range, and not repeated, 
therefore level 3 OC was out of control for the day’ 

- ‘09/01/2015 to 09/13/2015: all level 3 control results are outside the manufacturer's 
recommended range (negative bias)’ 

- ‘10/01/2015 - 10/31/2015: all level 3 control results are outside the manufacturer’s 
recommended range (negative bias)’ 

- ‘All 3 levels of OC were not run on 11/04/2015, 11/05/2015, 11/07/2015, 11/08/2015, 
11/09/2015, 11/10/2015, 11/12/2015, 11/14/2015, and 11/15/2015’ [The laboratory 
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provided no information as to whether patient HCG specimens were tested on the 
November dates listed. ] 


Yet, the laboratory concluded: ‘Results for urine HCG are quantitative with a cut off of 30 
mUlml. Because of the nature of this assay no correct reports are required.’ 


Since OC failures may indicate possible test system problems and it is unknown as to how the 
HCG (human chronic gonadotropin) test results were used and interpreted, we question the 
laboratory's conclusion and the accuracy and reliability of any patient HCG test results 
reported on the July, September, October, and November dates listed.” 


Theranos Response to Statement 1 — D5793 Finding #3: The laboratory has conducted a 
further review of this data and has revised its patient assessment analysis, as reflected in the 
enclosed documentation. (Ex. AA, Tab 8). 


D5793 Finding #3 — March 18 Letter Statement 2: “Zo ensure the deficient practice does not 
recur, the laboratory indicated that quarterly audits will be performed and suggested that the 
audits results would be reviewed within the laboratory's OMPI Program. However, the 
laboratory did not establish the procedure by which these quarterly audits are to be conducted. 
In its submission, the laboratory indicates that a “tracer audit may [emphasis added] be used,” 
but did not provided a protocol for a “tracer audit,” the means by which a “tracer audit” would 
be documented, and whether the results of a “tracer audit” would be the information reviewed 
by the QMPI Program.” 


Theranos Response to Statement 2 — D5793 Finding #3: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-8C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 F1. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]oth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB, Tab 7 § 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tabs 7B-7C). 


Finding #4 
D5793 Finding #4 — March 18 Letter Statement 1: “Jn the submission, the laboratory states: 
- ‘Anti-HBs is a qualitative assay with a positive cutoff of 11 so no corrected reports are 
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required. ’ 
- ‘Qualitative assay will not require corrected reports to be issued’ 


We are unclear as to how the laboratory came to these conclusions and the submission does not 
contain any documentation to support these statements.” 


Theranos Response to Statement 1 — D5793 Finding #4: The laboratory has conducted a 
further review of this data and has revised its patient assessment analysis, as reflected in the 
enclosed documentation. (Ex. AA, Ex. 9). 


D5793 Finding #4 — March 18 Letter Statement 2: “Nevertheless, to ensure the deficient 
practice does not recur, the laboratory indicated that quarterly audits will be performed and 
suggested that the audits results would be reviewed within the laboratory's OMPI Program. 
However, the laboratory did not establish the procedure by which these quarterly audits are to 
be conducted. In its submission, the laboratory indicates that a “tracer audit may [emphasis 
added] be used,” but did not provided a protocol for a “tracer audit,” the means by which a 
“tracer audit” would be documented, and whether the results of a “tracer audit” would be the 
information reviewed by the OMPI Program.” 


Theranos Response to Statement 2 — D5793 Finding #4: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-8C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 Fl. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b oth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB, Tab 7 § 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tabs 7B-7C). 


Finding #5 


D5793 Finding #5 — March 18 Letter Statement 1: “Jn the submission, the laboratory states: 
‘Corrected [LH] reports to be sent for patient samples analyzed for the following time period: 
7/1/2015-7/8/2015; all patient samples analyzed in 8/2015 and 9/2015.’ However, later in the 
submission, the laboratory concluded that ‘[b]ased on this review, 0 corrected reports are being 
issued.’ From these conflicting statements, it is unclear whether the laboratory addressed patient 
outcomes. The laboratory provided no documentation to indicate that corrected LH (luteinizing 
hormone) patient reports were issued.” 
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Theranos Response to Statement 1 — D5793 Finding #5: The laboratory’s patient 
assessment analysis for this finding appears to have included an inadvertent copy and paste 
mistake. The laboratory has enclosed its revised analysis, which explains why it has issued 
corrected reports for certain patients having the potential to be affected. (Ex. AA, Tab 10). 
Many corrected reports have been transmitted, and the remainder are being transmitted. (Ex. 
KK). Transmission will be complete by March 31, 2016. The remainder of the transmittals 
and confirmations of receipt will be provided to CMS under separate cover. 


D5793 Finding #5 — March 18 Letter Statement 2: “Zo ensure the deficient practice does not 
recur, the laboratory indicated that quarterly audits will be performed and suggested that the 
audits results would be reviewed within the laboratory's OMPI Program. However, the 
laboratory did not establish the procedure by which these quarterly audits are to be conducted. 
In its submission, the laboratory indicates that a “tracer audit may [emphasis added] be used,” 
but did not provided a protocol for a “tracer audit,” the means by which a “tracer audit” would 
be documented, and whether the results of a “tracer audit” would be the information reviewed 
by the QMPI Program.” 


Theranos Response to Statement 2 — D5793 Finding #5: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-8C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 F1. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]oth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB, Tab 7 § 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tabs 7B-7C). 


Finding #6 
D5793 Finding #6 — March 18 Letter Statement 1: “/n the submission, the laboratory states: 
“All patient results reported> 472 U/ml will need corrected reports.” 
- “Based on this review, 122 corrected [CA-125] reports are being issued to reflect the 
verified reportable range.” 


The laboratory provided no documentation to indicate corrected reports were generated and 


issued. 
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Theranos Response to Statement 1 — D5793 Finding #6: Many corrected reports have 
been transmitted, and the remainder are being transmitted. (Ex. KK). Transmission will be 
complete by March 31, 2016. The remainder of the transmittals and confirmations of receipt 
will be provided to CMS under separate cover. 


D5793 Finding #6 — March 18 Letter Statement 2: “Zo ensure the deficient practice does not 
recur, the laboratory indicated that quarterly audits will be performed and suggested that the 
audits results would be reviewed within the laboratory’s OMPI Program. However, the 
laboratory did not establish the procedure by which these quarterly audits are to be conducted. 
In its submission, the laboratory indicates that a “tracer audit may [emphasis added] be used,” 
but did not provided a protocol for a “tracer audit,” the means by which a “tracer audit” would 
be documented, and whether the results of a “tracer audit” would be the information reviewed 
by the OMPI Program.” 


Theranos Response to Statement 2 — D5793 Finding #6: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-8C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 Fl. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]oth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB, Tab 7 § 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tabs 7B-7C). 


Finding #7 


D5793 Finding #7 — March 18 Letter Statement 1: “7he submission references “Ex. H, Tabs 
1, 2-5, 16, 17-20, 21, 22 - 25, 26, 27 - 30, 36, 37 - 40,46, 47 - 50, 56, and 57 - 60.” We found 
no such references in the materials provided to CMS.” 


Theranos Response to Statement 1 — D5793 Finding #7: The citation references to Ex. H, 
Tabs 1, 2 - 5, 16, 17 - 20, 21, 22 - 25, 26, 27 - 30, 36, 37 - 40,46, 47 - 50, 56, and 57 — 60 
were inadvertent transcription errors, and the inclusion of these blank tabs in the materials 
was an inadvertent copying error. 


D5793 Finding #7 — March 18 Letter Statement 2: “7hroughout Ex. H, the submission 
includes lists of patient specimen accession numbers for which the laboratory intended to issue 
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corrected test reports. The laboratory provided no documentation to indicate corrected reports 
were generated and issued.” 


Theranos Response to Statement 2 — D5793 Finding #7: Many corrected reports have 
been transmitted, and the remainder are being transmitted. (Ex. KK). Transmission will be 
complete by March 31, 2016. The remainder of the transmittals and confirmations of receipt 
will be provided to CMS under separate cover. 


D5793 Finding #7 — March 18 Letter Statement 3: “We found no information addressing how 
the laboratory will ensure the timely review of the effectiveness of any actions taken. Although 
the laboratory indicated that quarterly audits will be performed to ensure compliance with this 
deficient practice, it is unclear how quarterly audits will timely address the issues cited under 
this deficiency. Nevertheless, the laboratory did not establish the procedure by which these 
quarterly audits are to be conducted. In its submission, the laboratory indicates that a “tracer 
audit may [emphasis added] be used,” but did not provided a protocol for a “tracer audit,” the 
means by which a “tracer audit” would be documented, and whether the results of a “tracer 
audit” would be the information reviewed by the OMPI Program.” 


Theranos Response to Statement 3 — D5793 Finding #7: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-8C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 Fl. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]oth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB, Tab 7 § 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tabs 7B-7C). 


Finding #8 


D5793 Finding #8 — March 18 Letter Statement 1: “Zhe submission references ‘Lx. B, Tabs 
51 and 53-55, and Ex. H, Tabs 1,2 - 5, 17 - 20, 26, 27 -30, 36, 37 - 40, 46, 47 - 50, 56, and 57 - 
60.’ We found no such references in the materials provided to CMS.” 


Theranos Response to Statement 1 — D5793 Finding #8: The citation references to Ex. B, 
Tabs 51 and 53-55, and Ex. H, Tabs 1,2 - 5, 17 - 20, 26, 27 -30, 36, 37 - 40, 46, 47 - 50, 56, 
and 57 - 60 were inadvertent transcription errors, and the inclusion of these blank tabs in the 
materials was an inadvertent copying error. 
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D5793 Finding #8 — March 18 Letter Statement 2: “Zhe laboratory submitted a new OC 
procedure (I:x. A, Tab 1) which requires OC Pass/Ivail Criteria. Specifically, Section 8.2.1.1.a - 
c. provides that OC must meet certain criteria as well as *... the required Westgard rule pass 
criteria.’ Westgard rules included a “10x” rule for rejecting OC when 10 consecutive control 
measurements fall on one side of the mean. We note that on page 17 of 19 (Ex. A, Tab 6) in the 
updated protocol that the “10x” rule was included as part of the Westgard rules which the lab 
must follow. The lab provided no documentation indicating that the “10x” rule was evaluated as 
part of its review... Because the laboratory has not shown whether it can follow its own OC 
protocols, it is unclear whether the laboratory’s quality assessment mechanisms can monitor the 
laboratory's corrective actions and ensure this deficient practice does not recur.” 


Theranos Response to Statement 2 — D5793 Finding #8: The laboratory has extended this 
analysis to evaluate global quality control indicators, as reflected in the attached 
documentation. (Ex. AA, Tab 11). As a result of that analysis, the laboratory has voided 
additional results for certain chemistry assays. Many corrected reports have been 
transmitted, and the remainder are being transmitted. (Ex. KK). Transmission will be 
complete by March 31, 2016. The remainder of the transmittals and confirmations of receipt 
will be provided to CMS under separate cover. 


The laboratory is capable of ensuring that the deficient practice in D5793 Finding #8 does 
not recur. Significantly, the laboratory has brought in an entirely new leadership team that is 
directly involved with implementing and monitoring the laboratory’s new procedures. The 
new laboratory director, Dr. Kingshuk Das, began working for the laboratory full-time as of 
March 14, 2016. Dr. Das is also one of the laboratory’s technical supervisors, which further 
reflects his level of involvement in the laboratory. In addition, the laboratory has hired 
Donald Tschirhart, M.D to be a full-time co-director (under California regulations) and a 
full-time clinical consultant. Dr. Tschirhart has over 15 years of experience as a clinical 
laboratory director, and is a valuable part of the laboratory’s new leadership and oversight 
team. The laboratory’s new leadership team has installed an entirely redesigned QMPI 
Program, as well as robust audit procedures that required monthly and quarterly audits. 


D5793 Finding #8 — March 18 Letter Statement 3: “Jn Ex. B and Ex. H, the submission 
includes lists of patient specimen accession numbers for which the laboratory intended to issue 
corrected test reports. The laboratory provided no documentation to indicate corrected reports 
were generated and issued.” 


Theranos Response to Statement 3 — D5793 Finding #8: Many corrected reports have 
been transmitted, and the remainder are being transmitted. (Ex. KK). Transmission will be 
complete by March 31, 2016. The remainder of the transmittals and confirmations of receipt 
will be provided to CMS under separate cover. 
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Finding #9 


D5793 Finding #9 — March 18 Letter Statement 1: “7he laboratory did not submit any 
documentation in the referenced exhibits related to an updated protocol for Alternative 
Assessment Procedures (AAP); therefore, we have concluded that the current protocol applies. 
The laboratory’s submission did not include any documentation to explain why the laboratory 
did not follow its AAP protocol for the Theranos Proprietary System. Because the laboratory 
has not shown whether it can follow its own AAP protocols, it is unclear whether the 
laboratory's quality assessment mechanisms can monitor the laboratory’s corrective actions and 
ensure this deficient practice does not recur.” 


Theranos Response to Statement 1 — D5793 Finding #9: There appears to have been a 
miscommunication. The TPS 3.5 was fully retired in early-August 2015. In addition, the 
laboratory stopped using its other LDTs, and does not have plans to resume using them as it 
transitions to compliance with FDA quality systems requirements for its LDTs and 
introduces next generation technologies. Accordingly, the AAP protocol that was used for 
tests run on these instruments in 2014 and 2015 is not in use. 


Furthermore, although alternative proficiency testing is not required for any test that is in use 
in the lab, the laboratory’s revised proficiency testing procedure includes new procedures for 
alternative proficiency assessment. (Ex. BB, Tab 3 § 9.2.4). These alternative proficiency 
assessment procedures were in the version of the proficiency testing procedure included with 
our submission, and are also in the revised proficiency testing procedure enclosed with this 
letter. (Ex. BB, Tab 3). Training on the revised proficiency testing procedure has occurred. 
(Ex. BB, Tabs 3B-3C). The QMPI procedures state that the “meeting standing agenda” 
includes a discussion of analytical quality metrics including proficiency testing results and 
overall performance. (Ex. BB, Tab 7 § 7.5.2.5). 


The laboratory is capable of ensuring that the deficient practice in DS793 Finding #9 does 
not recur. Significantly, the laboratory has brought in an entirely new leadership team that is 
directly involved with implementing and monitoring the laboratory’s new procedures. The 
new laboratory director, Dr. Kingshuk Das, began working for the laboratory full-time as of 
March 14, 2016. Dr. Das is also one of the laboratory’s technical supervisors, which further 
reflects his level of involvement in the laboratory. In addition, the laboratory has hired 
Donald Tschirhart, M.D to be a full-time co-director (under California regulations) and a 
full-time clinical consultant. Dr. Tschirhart has over 15 years of experience as a clinical 
laboratory director, and is a valuable part of the laboratory’s new leadership and oversight 
team. The laboratory’s new leadership team has installed an entirely redesigned QMPI 
Program, as well as robust audit procedures that required monthly and quarterly audits. 
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Finding #10 


D5793 Finding #10 — March 18 Letter Statement 1: Ex. 4, Zab 9 of the submission contains 
the protocol “Method Validation.” Section 8.4 of this protocol, “Reference Intervals,” states: 


The reference intervals are established from a selected reference population. Typically, 
when both decreased and increased levels are clinically significant, it is defined as the 
interval between and including the lower and upper reference limits of the studies 
population. This is calculated as the central 95 percentile of the population distribution 
where the lower and upper reference limits are demarcated as the 2.5th and 97.5th 
percentiles of the underlying distribution of values ... The reference intervals are used to 
establish expected level in health “normal” individuals. Values outside the reference 


” 


range are considered “abnormal” or “high” or “low. 
In Ex. E, Tab 1, the laboratory states: 


[The] laboratory director decided to use medical decision limits based on national 
consensus. Accordingly, the following decision limits were displayed on all lab reports 
for Vitamin D. As noted in CLSI EP28-A3, “when decision limits determined by national 
or worldwide consensus exist, these limits, rather than reference intervals should be 
reported. 


In Ex. E of the submission, the laboratory provided the following “medical decision limits:” 


Deficiency: <20.0 ngmL 
Insufficiency: 20.0 - 29.0 ng/mL 
Sufficiency: 30.0 - 100.0 ng/mL 
Possible Toxicity: 100.0 ng/mL 


We were unclear as to the laboratory’s definition of “medical decision limits.” No definition of 
this phrase was found in Section 3 - Definitions of the “Method Validation” protocol. In 
addition, no reference to CLSI EP28-A3 was found in the submitted protocol. We also note that 
the submitted protocol does not include a reference to “medical decision limits” as a criterion 
for determining reference range. 


Theranos Response to Statement 1 — D5793 Finding #10: DS793 Finding #10 relates to 
the TPS 3.5. The TPS 3.5 was fully retired in early-August 2015. The laboratory’s revised 
method verification procedure references a “medical decision level,” which is defined in the 
procedure as “[t]he concentration of analyte in blood or bodily fluid sample being analyzed at 
which some medical action is indicated for proper patient care. There may be several medical 
decision limits for a given analyte.” (Ex. BB, Tab 5 § 3.9). In addition, this revised 
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procedure references CLSI EP28-A3 (Ex. BB, Tab 5 § 10.6), which states: “for some 
analytes, reference intervals have been replaced by decision limits [emphasis in original], 
established by national (or international) consensus. As examples, consider cholesterol and 
glycated hemoglobin.” Training on the laboratory’s revised method verification procedure 
has occurred. (Ex. BB, Tabs S5B-5C). 


D5793 Finding #10 — March 18 Letter Statement 2: Since the laboratory provided no 
definition of “critical medical decision level,” no information regarding the laboratory’s 
disparity between the reference range (9.3 - 47.9 ng/mL) in the validation documentation and 
raw data reports, and no explanation as to how a Vitamin D level could be both normal and 
deficient or insufficient, it is unclear as to whether the laboratory has followed its own “Method 
Validation” protocol set forth at Section 8.4..... Because the laboratory has not shown whether it 
could follow its own validation protocols, it is unclear whether the laboratory’s quality 
assessment mechanisms can monitor the laboratory’s corrective actions and ensure this deficient 
practice does not recur. 


Theranos Response to Statement 2 — D5793 Finding #10: D5793 Finding #10 concerns 
the TPS 3.5. There appears to have been a miscommunication because the TPS 3.5 was fully 
retired in early-August 2015. Accordingly, the laboratory has not re-validated any assay on 
the TPS 3.5. References to validation data in documents enclosed with the submission are to 
validation reports prepared before the survey began and before the laboratory’s revised 
procedures were made effective (those reports were either reviewed by CMS or available to 
CMS during the survey). 


Because there is a national consensus for the decision levels for Vitamin D, the laboratory 
was correct in using the national consensus instead of a reference interval established through 
a validation study. CLSI EP28-A3 states: “for some analytes, reference intervals have been 
replaced by decision limits [emphasis in original], established by national (or international) 
consensus. As examples, consider cholesterol and glycated hemoglobin.” 


The laboratory’s revised method verification procedure references a “medical decision 
level,” which is defined in the procedure as “[t]he concentration of analyte in blood or bodily 
fluid sample being analyzed at which some medical action is indicated for proper patient 
care. There may be several medical decision limits for a given analyte.” (Ex. BB, Tab 5 § 
3.9). In addition, this revised procedure references CLS] EP28-A3. (Ex. BB, Tab 5 § 10.6). 
Training on the laboratory’s revised method verification procedure has occurred. (Ex. BB, 
Tabs 5B-SC). 


Additionally, the laboratory is capable of ensuring that the deficient practice in D5793#10 
does not recur. Significantly, the laboratory has brought in an entirely new leadership team 
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that is directly involved with implementing and monitoring the laboratory’s new procedures. 
The new laboratory director, Dr. Kingshuk Das, began working for the laboratory full-time as 
of March 14, 2016. Dr. Das is also one of the laboratory’s technical supervisors, which 
further reflects his level of involvement in the laboratory. In addition, the laboratory has 
hired Donald Tschirhart, M.D to be a full-time co-director (under California regulations) and 
a full-time clinical consultant. Dr. Tschirhart has over 15 years of experience as a clinical 
laboratory director, and is a valuable part of the laboratory’s new leadership and oversight 
team. The laboratory’s new leadership team has installed an entirely redesigned QMPI 
Program, as well as robust audit procedures that required monthly and quarterly audits. 


D5793 Finding #10 — March 18 Letter Statement 3: “Zhroughout Ex. E, the laboratory 
provided a list of patient specimen accession numbers for which the laboratory ‘identified 
reports,’ but did not specify or submit documentation to indicate whether corrected reports were 
generated and issued.” 


Theranos Response to Statement 2 — D5793 Finding #10: This deficiency concerns the 
TPS 3.5, which was one of the Theranos proprietary technologies in use in 2014 and 
2015. The laboratory began phasing out the TPS 3.5 in December 2014, and it was fully 
retired in early-August 2015. 


The laboratory values CMS’s feedback, and takes it very seriously. The laboratory has 
conducted a thorough re-review of QC data for each assay run on the TPS 3.5 from January 
1, 2014 until the TPS 3.5 was fully retired in early-August 2015. As explained further in the 
attached patient assessment analysis, the laboratory is not satisfied with its old quality 
assessment program’s ability to effectively flag and promptly address QC imprecision, QC 
failures, and QC trends on the TPS 3.5. (Ex. AA, Tab 3). The laboratory has new leadership 
and a new QMPI Program with robust quality monitoring and program improvement 
procedures. Based on its re-review of QC data for the TPS 3.5, the laboratory has 
determined that its prior QA program failed to satisfactorily address these types of QC issues 
for assays run on the TPS 3.5 in 2014 and 2015. As corrective action, the laboratory has, out 
of an extreme abundance of caution, voided results reported for assays run on the TPS 3.5 in 
2014 and 2015. (Ex. AA, Tab 3). Many corrected reports have been transmitted, and the 
remainder are being transmitted. (Ex. KK). Transmission will be complete by March 31, 
2016. The remainder of the transmittals and confirmations of receipt will be provided to 
CMS under separate cover. 


DS5801 


D5801 — March 18 Letter Statement 1: “Zhe submission references “Ex. I, Tabs 2-6." We 
located these tabs, but found no documentation in Tabs 2, 5 and 6.” 
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Theranos Response to Statement 1 — D5801: There was an inadvertent copying error for 
Exhibit I to the initial submission. The laboratory has enclosed the applicable documents 
with this letter, as well as additional documents related to the laboratory’s broader analysis of 
the PT/INR test over the entire time it was run on this Siemens BCS XP instrument. (Ex. 
GG). 


D5801 — March 18 Letter Statement 2: ‘Jn the submission, the laboratory states: 


Before PT/INR testing resumes, the lab will also prepare a revised assay-specific 
procedure for PT/INR to reinforce that the International Normalized Ratio (INR) must 
be calculated accurately prior to reporting patient test results. The relevant testing 
personnel will be required to demonstrate competency to ensure the practice is 
consistent with these procedures. 


We note that the laboratory provided no documentation to indicate that the inaccurate 
calculations were reviewed or that INR calculation using different lot numbers of Dade Innovin 
were reviewed for accuracy. In addition, we found that the laboratory did not provide 
documentation to ensure that P1/1NR results reported from calculations were, and would 
continue to be, accurate.” 


Theranos Response to Statement 2 — D5801: As noted above, the laboratory stopped 
running PT/INR tests on the Siemens BCS XP on September 17, 2015, and has not run any 
PT/INR tests since then. The laboratory has reviewed the inaccurate calculation and has 
reviewed the INR calculations for different lot numbers of Dade Innovin that were used with 
this Siemens BCS XP instrument from the time it was put in use in October 2014. As a result 
of this review, the laboratory has determined that certain INR calculations for prior lots were 
not accurate because the laboratory did not use the correct MNPT and ISI figures for those 
calculations. Based on this finding and other findings in the revised PT/INR patient 
assessment that is enclosed with this letter, the laboratory has voided all PT/INR results 
reported from October 2014 through September 2015. (Ex. AA, Tab 7). Many corrected 
reports have been transmitted, and the remainder are being transmitted. (Ex. 

KK). Transmission will be complete by March 31, 2016. The remainder of the transmittals 
and confirmations of receipt will be provided to CMS under separate cover. 


The laboratory is capable of ensuring that the deficient practice in D5801 does not recur. 
Significantly, the laboratory has brought in an entirely new leadership team that is directly 
involved with implementing and monitoring the laboratory’s new procedures. The new 
laboratory director, Dr. Kingshuk Das, began working for the laboratory full-time as of 
March 14, 2016. Dr. Das is also one of the laboratory’s technical supervisors, which further 
reflects his level of involvement in the laboratory. In addition, the laboratory has hired 
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Donald Tschirhart, M.D to be a full-time co-director (under California regulations) and a 
full-time clinical consultant. Dr. Tschirhart has over 15 years of experience as a clinical 
laboratory director, and is a valuable part of the laboratory’s new leadership and oversight 
team. The laboratory’s new leadership team has installed an entirely redesigned QMPI 
Program, as well as robust audit procedures that required monthly and quarterly audits. 


D5801 — March 18 Letter Statement 2: ‘Zo ensure the deficient practice does not recur, the 
laboratory indicated that quarterly audits will be performed and suggested that the audits results 
would be reviewed within the laboratory’s QMPI Program. However, the laboratory did not 
establish the procedure by which these quarterly audits are to be conducted. In its submission, 
the laboratory indicates that a “tracer audit may [emphasis added] be used,” but did not 
provided a protocol for a “tracer audit,” the means by which a “tracer audit” would be 
documented, and whether the results of a “tracer audit” would be the information reviewed hy 
the OMPI Program.” 


Theranos Response to Statement 2 — D5801: The laboratory has revised its audit 
procedures to establish the procedures by which audits are to be conducted. (Ex. BB, Tab 8). 
Training on these procedures has occurred. (Ex. BB, Tabs 8B-8C). The revised procedures 
establish that tracer audits must occur monthly; provide the protocol for a tracer audit; and 
require that tracer audits be documented using the tracer audit form, CL FRM 00046 F2. 
(Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a quarterly audit to 
cover the quality management systems; provide the protocol for quarterly audits; and require 
that quarterly audits are documented using the audit record form, CL QOP 00046 F1. (Ex. 
BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear that “[b]oth 
tracer and quarterly audits are reviewed, analyzed and presented based on findings and 
analysis” during the QMPI Program meetings. (Ex. BB, Tab 7 § 7.5). Training on the 
revised QMPI procedures has occurred. (Ex. BB, Tabs 7B-7C). 


D5805 


D5805 — March 18 Letter Statement 1: “Zhe submission references “Ex. I, Tabs 2-6." We 
located these tabs, but found no documentation in Tabs 2, 5 and 6.” 


Theranos Response to Statement 1 — D5805: There was an inadvertent copying error for 
Exhibit I to the initial submission. The laboratory has enclosed the applicable documents 
with this letter, as well as additional documents related to the laboratory’s broader analysis of 
the PT/INR test over the entire time it was run on this Siemens BCS XP instrument. (Ex. 
GG). 


D5805 — March 18 Letter Statement 2: /n its submission, the laboratory states: ‘The lab 
revised its patient reports during the survey so that the interpretive note appears only under the 
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heading for patients with therapy.’ However, no updated forms were shown to the surveyor 
during the survey and the laboratory’s submission does not include documentation of the 
updated reports. 


D5805 — March 18 Letter Statement 2: As indicated above, the laboratory performed a 
patient assessment for PT/INR tests run on this BCS XP instrument with reagent lots that 
were used prior to the reagent lot identified in CMS’s Form 2567. Based on that review, the 
laboratory has voided all PT/INR results from the time this BCS XP instrument was put in 
use in October 2014. (Ex. AA, Tab 7). Many corrected reports have been transmitted, and 
the remainder are being transmitted. (Ex. KK). Transmission will be complete by March 31, 
2016. The remainder of the transmittals and confirmations of receipt will be provided to 
CMS under separate cover. 


D5805 — March 18 Letter Statement 3: “Zo ensure the deficient practice does not recur, the 
laboratory indicated that quarterly audits will be performed and suggested that the audits results 
would be reviewed within the laboratory’s OMPI Program. However, the laboratory did not 
establish the procedure by which these quarterly audits are to be conducted. In its submission, 
the laboratory indicates that a “tracer audit may [emphasis added] be used,” but did not 
provided a protocol for a “tracer audit,” the means by which a “tracer audit” would be 
documented, and whether the results of a “tracer audit” would be the information reviewed by 
the OMPI Program.” 


Theranos Response to Statement 3 — D5805: The laboratory has revised its audit 
procedures to establish the procedures by which audits are to be conducted. (Ex. BB, Tab 8). 
Training on these procedures has occurred. (Ex. BB, Tabs 8B-8C). The revised procedures 
establish that tracer audits must occur monthly; provide the protocol for a tracer audit; and 
require that tracer audits be documented using the tracer audit form, CL FRM 00046 F2. 
(Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a quarterly audit to 
cover the quality management systems; provide the protocol for quarterly audits; and require 
that quarterly audits are documented using the audit record form, CL QOP 00046 F1. (Ex. 
BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear that “[b]oth 
tracer and quarterly audits are reviewed, analyzed and presented based on findings and 
analysis” during the QMPI Program meetings. (Ex. BB, Tab 7 § 7.5). Training on the 
revised QMPI procedures has occurred. (Ex. BB, Tabs 7B-7C). 


D5821 


D5821 — March 18 Letter Statement 1: “Zhe submission references “Ex. I, Tabs 2-6.” We 
located these tabs, but found no documentation in Tabs 2, 5 and 6.” 
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Theranos Response to Statement 1 — D5821: There was an inadvertent copying error for 
Exhibit I to the initial submission. The laboratory has enclosed the applicable documents 
with this letter, as well as additional documents related to the laboratory’s broader analysis of 
the PT/INR test over the entire time it was run on this Siemens BCS XP instrument. (Ex. 
GG). 


D5821 — March 18 Letter Statement 2: “Jn x. /, the laboratory provided lists of patient 
specimen accession numbers for which the laboratory intended to issue corrected test reports. 
The laboratory provided no documentation to indicate corrected reports were issued. “We also 
note that the Patient Impact Assessment (Ex. I, Tab 1) states that ‘remedial action was taken on 
9/25/15,’ but ‘corrected reports were issued beginning on 11/10/15 and completed on 11/12/15.’ 
There was no explanation as to why there was such a long period of time between the remedial 
action and issuing corrected reports.” 


Theranos Response to Statement 2 — D5821: All of these corrected reports for PT/INR, 
along with documentation confirming that they were sent, are enclosed with this letter. (Ex. 
GG, Tab 7). On September 22 or 23, CMS identified a number of issues related to PT/INR 
tests during the period April to September 2015. The laboratory undertook a thorough 
investigation. Given the number of issues to review, the investigation took time to complete. 
Completion of the investigation was also hindered by the absence of a highly qualified, full- 
time laboratory director. The laboratory subsequently hired its new lab director, Dr. 
Kingshuk Das, who is highly qualified and has extensive clinical laboratory experience. As 
of March 14, 2016, Dr. Das joined Theranos full-time at the Newark, California laboratory. 


D5821 — March 18 Letter Statement 3: “Zo ensure the deficient practice does not recur, the 
laboratory indicated that quarterly audits will be performed and suggested that the audits results 
would be reviewed within the laboratory’s QMPI Program. However, the laboratory did not 
establish the procedure by which these quarterly audits are to be conducted. In its submission, 
the laboratory indicates that a “tracer audit may [emphasis added] be used,” but did not 
provided a protocol for a “tracer audit,” the means by which a “tracer audit” would be 
documented, and whether the results of a “tracer audit” would be the information reviewed by 
the OMPI Program.” 


Theranos Response to Statement 2 — D5821: The laboratory has revised its audit 
procedures to establish the procedures by which audits are to be conducted. (Ex. BB, Tab 8). 
Training on these procedures has occurred. (Ex. BB, Tabs 8B-8C). The revised procedures 
establish that tracer audits must occur monthly; provide the protocol for a tracer audit; and 
require that tracer audits be documented using the tracer audit form, CL FRM 00046 F2. 

(Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a quarterly audit to 
cover the quality management systems; provide the protocol for quarterly audits; and require 
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that quarterly audits are documented using the audit record form, CL QOP 00046 F1. (Ex. 
BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear that “[b]oth 
tracer and quarterly audits are reviewed, analyzed and presented based on findings and 
analysis” during the QMPI Program meetings. (Ex. BB, Tab 7 § 7.5). Training on the 
revised QMPI procedures has occurred. (Ex. BB, Tabs 7B-7C). 


D6076 


The March 18 Letter states: “See our reviews of D6083, D6085, D6086, D6093, D6094, D6098, 
and D6102.”" Accordingly, we refer you to our responses to those reviews. 


D6079 


The March 18 Letter states that “[b]ased on the laboratory’s submission, this requirement is 
met.” 


D6083 


The March 18 Letter states: “See our reviews of D5413 and D5791.” Accordingly, we refer you 
to our responses to those reviews. 


D6085 


The March 18 Letter states: “See our review of D6115.” Accordingly, we refer you to our 
responses to your review of D6115. 


D6086 


D6086 Finding #1—March 18 Letter Statement 1: “Zhe submission references ‘Ex. A, Tab 9, 
F-l” and “Ex. A, Tab 10, F-1.’ We found no such references in the materials provided to CMS.” 


Theranos Response to Statement | in D6086 Finding #1: The citation references to Ex. A, 
Tab 9, F-1 and Ex. A, Tab 10, F-1 were inadvertent transcription errors and should not have 
been included. 


D6086 Finding #1 — March 18 Letter Statement 2: “7hroughout Ex. E, Tab 1, the laboratory 
provided lists of patient specimen accession numbers for which the laboratory intended to issue 
corrected test reports. The laboratory provided no documentation to indicate corrected reports 
were generated and issued.” 


Theranos Response to Statement 2 in D6086 Finding #1: This deficiency concerns the 
TPS 3.5, which was one of the Theranos proprietary technologies in use in 2014 and 
2015. The laboratory began phasing out the TPS 3.5 in December 2014, and it was fully 
retired in early-August 2015. 
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The laboratory values CMS’s feedback, and takes it very seriously. The laboratory has 
conducted a thorough re-review of QC data for each assay run on the TPS 3.5 from January 
1, 2014 until the TPS 3.5 was fully retired in early-August 2015. As explained further in the 
attached patient assessment analysis, the laboratory is not satisfied with its old quality 
assessment program’s ability to effectively flag and promptly address QC imprecision, QC 
failures, and QC trends on the TPS 3.5. (Ex. AA, Tab 3). The laboratory has new leadership 
and a new QMPI Program with robust quality monitoring and program improvement 
procedures. Based on its re-review of QC data for the TPS 3.5, the laboratory has 
determined that its prior QA program failed to satisfactorily address these types of QC issues 
for assays run on the TPS 3.5 in 2014 and 2015. As corrective action, the laboratory has, out 
of an extreme abundance of caution, voided results reported for assays run on the TPS 3.5 in 
2014 and 2015. (Ex. AA, Tab 3). Many corrected reports have been transmitted, and the 
remainder are being transmitted. (Ex. KK). Transmission will be complete by March 31, 
2016. The remainder of the transmittals and confirmations of receipt will be provided to 
CMS under separate cover. 


D6086 Finding #1 — March 18 Letter Statement 3: We found no information addressing how 
the laboratory will ensure the accurate review of test method validation documentation, the issue 
cited in the deficiency. 


Theranos Response to Statement 3 in D6086 Finding #1: The TPS 3.5 was fully retired 
early-August 2015, and the laboratory is not using any LDTs. 


The laboratory has revised its method verification procedure to show how it will ensure the 
accurate review of test method verification documentation. (Ex. BB, Tab 5). As set forth in 
the revised procedures, the laboratory director is responsible for reviewing both the 
verification plan and the verification report. (Ex. BB, Tab 5 § 4.1). “The method 
verification procedure requires multiple reviews to include Quality to ensure protocols are 
followed.” (Ex. BB, Tab 7 § 7.1.3.3). 


Finding #2 


D6086 Finding #2—March 18 Letter Statement 1: “Zhe submission references ‘Ex. H, Tabs 1, 
2-5, 16, 17 - 20, 21, 22 - 25, 26, 27 - 30, 36, 37 - 40, 46, 47 - 50, 56, and 57 - 60.’ We found no 
such references in the materials provided to CMS.” 


Theranos Response to Statement 1 in D6086 Finding #2: The citation references to Ex. H, 
Tabs 1, 2-5, 16, 17 - 20, 21, 22 - 25, 26, 27 - 30, 36, 37 - 40, 46, 47 - 50, 56, and 57-60 were 
inadvertent. 
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D6086 Finding #2—March 18 Letter Statement 2: “Ex. H contains lists of patient specimen 
accession numbers for which the laboratory intended to issue corrected test reports. The 
laboratory provided no documentation to indicate corrected reports were generated and issued.” 


Theranos Response to Statement 2 in D6086 Finding #2: Many corrected reports have 
been transmitted, and the remainder are being transmitted. (Ex. KK). Transmission will be 
complete by March 31, 2016. The remainder of the transmittals and confirmations of receipt 
will be provided to CMS under separate cover. 


D6086 Finding #2—March 18 Letter Statement 3: “Af the time of the onsite survey, the 
laboratory's protocol ‘Master Validation Plan for Routine Chemistry Assays on Theranos 
Devices, ’ stated: *... for establishing the trueness or comparability to two procedures. . .at least 
50% of samples should be outside the reference interval.’ For five validation documents (ALT, 
BUN, calcium, glucose, and sodium testing using the Advia 1800), a review of the test results 
used by the laboratory to establish ‘the trueness or comparability of two procedures’ showed 
that the laboratory did not follow its established protocol and use “at least 50% of samples ... 
outside the reference interval. The submission includes the protocol ‘Method Validation, ’ at Ex. 
A, Tab 9, Section 8.1.1.1 (‘A method comparison and bias estimation design’), which states: ‘If 
the analyte has a critical medical decision level, at least 50% of the specimens must have analyte 
levels below this value and the remaining 50% above.’ We are unclear as to the laboratory's 
definition of ‘critical medical decision level.’ No definition of this phrase was found in Section 3 
- Definitions of the “Method Validation” protocol or in Ex. H.”"”" 


Theranos Response to Statement 3 in D6086 Finding #2: D6086 Finding #2 relates to the 
chemistry assays on the Siemens Advia 1800 instruments. The laboratory stopped using the 
Siemens Advia 1800 instruments before the survey began on September 22, 2015, and has 
not used them since then. 


The laboratory’s revised method verification procedure references a “medical decision 
level,” which is defined in the procedure as “[t]he concentration of analyte in blood or bodily 
fluid sample being analyzed at which some medical action is indicated for proper patient 
care. There may be several medical decision limits for a given analyte.” (Ex. BB, Tab 5 § 
3.9). In addition, this revised procedure references CLSI EP28-A3 (Ex. BB, Tab 5 § 10.6), 
which states: “for some analytes, reference intervals have been replaced by decision limits 
[emphasis in original], established by national (or international) consensus. As examples, 
consider cholesterol and glycated hemoglobin.” Training on the laboratory’s revised method 
verification procedure has occurred. (Ex. BB, Tabs SB-SC). 


D6086 Finding #2—March 18 Letter Statement 4: “/n Lx. H of the submission, the 
laboratory provided the following ‘assay validation’ summary: 
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Analyte Number of Specimens Used Specimen Value Range 
ALT 128 12-62 

BUN 128 9-2] 

Calcium 128 9.2 -10.2 

Glucose 128 65 - 142 

Sodium 128 134 -142 


Since the laboratory provided no definition of ‘critical medical decision level’ and no 
information regarding the laboratory’s panic or alert values for ALT, BUN, calcium, glucose, 
and sodium testing, it is unclear as to whether the laboratory has followed its own “Method 


theranos.com 


Validation” protocol at Section 8.1.1.1. Because the laboratory has not shown whether it can 


follow its own validation protocols, it is unclear whether the laboratory’s quality assessment 


mechanisms can monitor the laboratory’s corrective actions and ensure this deficient practice 


does not recur.” 


Theranos Response to Statement 4 in D6086 Finding #2: There appears to have been a 


miscommunication. The laboratory stopped using its Siemens Advia 1800 instruments for 
clinical patient testing before the survey began on September 22, 2015 and before the revised 


method verification procedures were effective, and has not used them since then. 
Accordingly, the laboratory has not re-validated any assay on the Siemens Advia 1800. 


The laboratory’s revised method verification procedure references a “medical decision 


level,” which is defined in the procedure as “[t]he concentration of analyte in blood or bodily 


fluid sample being analyzed at which some medical action is indicated for proper patient 
care. There may be several medical decision limits for a given analyte.” (Ex. BB, Tab 5 § 


3.9). In addition, this revised procedure references CLSI EP28-A3 (Ex. BB, Tab 5 § 10.6), 
which states: “for some analytes, reference intervals have been replaced by decision limits 
/emphasis in original], established by national (or international) consensus. As examples, 
consider cholesterol and glycated hemoglobin.” Training on the laboratory’s revised method 


verification procedure has occurred. (Ex. BB, Tabs SB-5C). 


The laboratory is capable of ensuring that the deficient practice in D6086 Finding #2 does 


not recur. Significantly, the laboratory has brought in an entirely new leadership team that is 
directly involved with implementing and monitoring the laboratory’s new procedures. The 

new laboratory director, Dr. Kingshuk Das, began working for the laboratory full-time as of 
March 14, 2016. Dr. Das is also one of the laboratory’s technical supervisors, which further 


reflects his level of involvement in the laboratory. In addition, the laboratory has hired 
Donald Tschirhart, M.D to be a full-time co-director (under California regulations) and a 
full-time clinical consultant. Dr. Tschirhart has over 15 years of experience as a clinical 
laboratory director, and is a valuable part of the laboratory’s new leadership and oversight 
team. The laboratory’s new leadership team has installed an entirely redesigned QMPI 
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Program, as well as robust audit procedures that required monthly and quarterly audits. With 
respect to method verification, the procedure now includes appropriate criteria for accuracy, 
precision, and other pertinent performance characteristics, and requires the lab director to 
review both the verification plan and verification report to ensure adherence to the criteria in 
the procedure. (Ex. BB, Tab 5 §§ 7.7, 8). 


Finding #3 


D6086 Finding #3—March 18 Letter Statement 1: “x. H contains lists of patient specimen 
accession numbers for which the laboratory intended to issue corrected test reports. The 
laboratory provided no documentation to indicate corrected reports were generated and issued.” 


Theranos Response to Statement 1 in D6086 Finding #3: Many corrected reports have 
been transmitted, and the remainder are being transmitted. (Ex. KK). Transmission will be 
complete by March 31, 2016. The remainder of the transmittals and confirmations of receipt 
will be provided to CMS under separate cover. 


D6086 Finding #3—March 18 Letter Statement 2: “A/ the time of the onsite survey, for five 
validation documents reviewed (ALT, BUN, calcium, glucose, and sodium testing using the Advia 
1800), laboratory summary information indicated that the laboratory obtained coefficient of 
variation (CV) results greater than CV’s established by the instrument manufacturer. That is, 
ALT, BUN, calcium, glucose, and sodium test results obtained by the laboratory were 
statistically less precise than what would be expected using the Advia 1800. The laboratory 
provided no explanation for and no information pertaining to any investigation as to why the 
laboratory’s test results were less precise. These validation documents were approved by the 
laboratory director as evidenced by his signature. In Ex. H of the submission, the laboratory 
provided summary data for ALT, BUN, calcium, glucose, and sodium similar to the CV data 
reviewed at the time of the onsite survey. Again, the laboratory provided no explanation for 
andor information pertaining to any investigation as to why the laboratory's test results were 
less precise. Because the laboratory has not shown whether its assay verification procedures are 
adequate to determine assay precision characteristics, we question whether the laboratory’s 
quality assessment mechanisms can monitor the laboratory’s corrective actions and ensure this 
deficient practice does not recur.” 


Theranos Response to Statement 2 in D6086 Finding #3: D6086 Finding #3 concerns the 
laboratory’s method for using capillary specimens to test for ALT, BUN, calcium, glucose 
and sodium using Advia 1800 instruments that the laboratory modified. There appears to 
have been a miscommunication. The laboratory stopped using its Siemens Advia 1800 
instruments for clinical patient testing before the survey began on September 22, 2015 and 
before the revised method verification procedures were effective, and has not used them 
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since then. Accordingly, the laboratory has not re-validated any assay on the Siemens Advia 
1800. 


Additionally, the CV figures from the manufacturer’s package insert are not relevant. Under 
42 C.F.R. § 1253(b)(1), a laboratory that “modifies an FDA-cleared or approved test system 
... must, before reporting patient results, establish for each test system the performance 
specifications for the following performance characteristics: (i) Accuracy, (ii) Precision, (iii) 
Analytical sensitivity, (iv) Analytical specificity to include interfering substances, (v) 
Reportable range of test results for the test system, (vi) Reference intervals (normal values), 
(vii) Any other performance characteristic required for test performance.” Under this 
regulation, the precision of an LDT is established solely by the laboratory. These 
regulations do not require a laboratory to compare the CV it establishes for an LDT — 
including an LDT that is based on a modification to an FDA-cleared or approved test system 
— with the %CV of “the predicate device.” Likewise, there is nothing in 42 C.F.R. 
§1253(b)(1) that requires a laboratory to document why the CV that it has established for an 
LDT is different from the CV for “a predicate device.” 


Finding #4 


D6086 Finding #4—March 18 Letter Statement 1: “Ex. H contains lists of patient specimen 
accession numbers for which the laboratory intended to issue corrected test reports. The 
laboratory provided no documentation to indicate corrected reports were generated and issued.” 


Theranos Response to Statement 1 in D6086 Finding #4: Many corrected reports have 
been transmitted, and the remainder are being transmitted. (Ex. KK). Transmission will be 
complete by March 31, 2016. The remainder of the transmittals and confirmations of receipt 
will be provided to CMS under separate cover. 


D6086 Finding #4—March 18 Letter Statement 2: “Af the time of the onsite survey, for four 
validation documents reviewed (ALT, BUN, calcium, and glucose testing using the Advia 1800), 
laboratory summary information indicated that the reference range determined by the 
laboratory’s testing differed from the reference range on the laboratory's test reports. The 
following was noted: 


Analyte Validation Document Reference Test Report Reference Range 
ALT Range 8-41 U/L 

BUN 0- 52 UIL 6 - 24 mg/dL 

Calcium 5.3 - 22.5 mg/dL. 8.3 - 10.6 mg/dL 

Glucose 8.18 -10.3 mg/dL 73 - 99 mg/dL 


64.0 - 112.3 mg/dL 
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In Ex. H of the submission, the laboratory provided the following reference ranges, some of 
which were still different than what appeared on the laboratory’s test reports: 


Analyte Reference Range 
ALT 8-41 UL 

BUN 6.2 - 22 mg/dl. 
Calcium 8.2 - 10.3 mgaL 
Glucose 64 - 112 mg/dl. 


Since the laboratory did not provide copies of any updated test reports in its submission, we are 
unable to determine whether the laboratory has corrected this deficient practice. Because the 
laboratory has not shown whether it had corrected this deficient practice, we are also unable to 
determine whether the laboratory’s quality assessment mechanisms can monitor the laboratory’s 
corrective actions and ensure this deficient practice does not recur.” 


Theranos Response to Statement 2 in D6086 Finding #4: As CMS observed in the Form 
2567, the validation reports for these assays were dated in April 2015. These validated 
assays went live on April 22, 2015. The laboratory has issued corrected reports updating the 
reference ranges for ALT, Calcium, and BUN tests run on the Siemens Advia 1800 on or 
after April 2015. Many corrected reports have been transmitted, and the remainder are being 
transmitted. (Ex. KK). Transmission will be complete by March 31, 2016. The remainder 
of the transmittals and confirmations of receipt will be provided to CMS under separate 
cover. As noted above, the laboratory stopped running these tests on the Siemens Advia 
1800 before the survey began on September, and has not run them since then. 


The laboratory has not issued corrected reports for Glucose because the reference range used 
in patient reports was consistent with the national consensus, which was the correct range to 
use. As noted above, the revised method verification procedures enclosed with this letter 
references CLSI EP28-A3, which provides that when decision limits determined by national 
or worldwide consensus exist, these limits, rather than reference intervals should be reported. 
(Ex. BB, Tab 5 § 10.6). 


The laboratory is capable of ensuring that the deficient practice in Do086 Finding #4 does 
not recur. Significantly, the laboratory has brought in an entirely new leadership team that is 
directly involved with implementing and monitoring the laboratory’s new procedures. The 
new laboratory director, Dr. Kingshuk Das, began working for the laboratory full-time as of 
March 14, 2016. Dr. Das is also one of the laboratory’s technical supervisors, which further 
reflects his level of involvement in the laboratory. In addition, the laboratory has hired 
Donald Tschirhart, M.D to be a full-time co-director (under California regulations) and a 
full-time clinical consultant. Dr. Tschirhart has over 15 years of experience as a clinical 
laboratory director, and is a valuable part of the laboratory’s new leadership and oversight 
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team. The laboratory’s new leadership team has installed an entirely redesigned QMPI 
Program, as well as robust audit procedures that required monthly and quarterly audits. 


Finding #5 


D6086 Finding #5 — March 18 Letter Statement 1: “Zhe submission references ‘Lx. G, Tabs 
31, 34, 36, 39, 41, 44, 46, 49, 51, 54, 56, 61, 64, 66, and 69.’ We found no documents within 
these tabs.” 


Theranos Response to Statement 1 in D6086 Finding #5: The citation reference to Ex. G, 
Tabs 31, 34, 36, 39, 41, 44, 46, 49, 51, 54, 56, 61, 64, 66, and 69 was an inadvertent 
transcription error, and the inclusion of these blank tabs in the materials was an inadvertent 
copying error. 


D6086 Finding #5 — March 18 Letter Statement 2: “A/ the time of the onsite survey, the 
laboratory maintained no documentation showing that the validation documents for two Siemens 
Advia 2120i instruments had been reviewed and approved by the laboratory director. In Ex. G 
of the submission, we again found no information to indicate the Siemens Advia 2120i validation 
documents have been reviewed and approved by the director. Because the laboratory has not 
shown whether it has corrected this deficient practice, we are also unable to determine whether 
the laboratory’s QA mechanisms can monitor the laboratory’s corrective actions and ensure this 
deficient practice does not recur.” 


Theranos Response to Statement 2 in D6086 Finding #5: There appears to be a 
miscommunication. The laboratory stopped using its Siemens Advia 2120i instruments for 
clinical patient testing on November 17, 2015, and has not used them since then. The 
laboratory does not intend to rely on those verifications to perform any tests on the Siemens 
Advia 2120i instruments in the future. As described above, the laboratory has put in place 
mechanisms that will effectively monitor its corrective action and ensure that deficient 
practices do not recur. With respect to method verification, the current procedures require 
the lab director to approve both the verification plan prior to the verification and the 
verification report. (Ex. BB, Tab 5 § 7.7). “The method verification procedure requires 
multiple reviews to include Quality to ensure protocols are followed.” (Ex. BB, Tab 7 § 
7.1.3.3). 


The laboratory is capable of ensuring that the deficient practice in D6086 Finding #5 does 
not recur. Significantly, the laboratory has brought in an entirely new leadership team that is 
directly involved with implementing and monitoring the laboratory’s new procedures. The 
new laboratory director, Dr. Kingshuk Das, began working for the laboratory full-time as of 
March 14, 2016. Dr. Das is also one of the laboratory’s technical supervisors, which further 
reflects his level of involvement in the laboratory. In addition, the laboratory has hired 
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Donald Tschirhart, M.D to be a full-time co-director (under California regulations) and a 
full-time clinical consultant. Dr. Tschirhart has over 15 years of experience as a clinical 
laboratory director, and is a valuable part of the laboratory’s new leadership and oversight 
team. The laboratory’s new leadership team has installed an entirely redesigned QMPI 
Program, as well as robust audit procedures that required monthly and quarterly audits. 


Finding #6 


D6086 Finding #6 — March 18 Letter Statement 1: “Zhe submission references ‘Ex. I, Tabs 2- 
6.’ We located these tabs, but found no documentation in Tabs 2, 5, and 6.” 


Theranos Response to Statement 1 — D6086 Finding #6: There was an inadvertent 
copying error for Exhibit I to the initial submission. The laboratory has enclosed the 
applicable documents with this letter, as well as additional documents related to the 
laboratory’s broader analysis of the PT/INR test over the entire time it was run on this 
Siemens BCS XP instrument. (Ex. GG). 


D6086 Finding #6 — March 18 Letter Statement 2: “See our review of D5413.” 


Theranos Response to Statement 2 — D6086 Finding #6: See the laboratory’s response to 
your review of D5413. 


D6086 Finding #6 — March 18 Letter Statement 3: “A/though the submitted protocol (Ex. A, 
Tab 31, § 8.1.2) included the requirement to review all new manufacturer package inserts and 
vendor notifications, the laboratory did not provide a response to the citation other than to state: 
‘Theranos did not have the documentation for the MNPT calculation done for lot #539280 in 
3/2015 before it was put in use for patient testing.’ It is unclear what investigation was 
performed to determine how the Mean Normal Prothrombin Time (MNPT) for Dade Innovin lot 
number 539280 was defined and entered into the Siemens BCS-XP..” 


Theranos Response to Statement 3 — D6086 Finding #6: The lab personnel responsible for 
MNPT for PT/INR tests run on the Siemens BCS XP at the time that this Dade Innovin lot 
was put in use are no longer with the laboratory, which has limited the laboratory’s ability to 
determine precisely what occurred. Based on the laboratory’s investigation, it appears that 
the laboratory would determine the MNPT with the assistance of Siemens. Siemens would 
then send the laboratory a letter containing both the MNPT and the ISI for the lot. The 
laboratory would then be responsible for entering the MNPT and ISI numbers for the lot into 
the BCS XP. The laboratory appears to have not properly entered the MNPT and ISI 
numbers for this lot and the laboratory failed to have sufficient oversight to identify the 
problem when it occurred. 
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As noted above, the laboratory stopped running PT/INR tests on September 17, 2015, and 
has not run any PT/INR tests since then. The laboratory is capable of ensuring that the 
deficient practice in D6086 Finding #6 does not recur. Significantly, the laboratory has 
brought in an entirely new leadership team that is directly involved with implementing and 
monitoring the laboratory’s new procedures. The new laboratory director, Dr. Kingshuk Das, 
began working for the laboratory full-time as of March 14, 2016. Dr. Das is also one of the 
laboratory’s technical supervisors, which further reflects his level of involvement in the 
laboratory. In addition, the laboratory has hired Donald Tschirhart, M.D to be a full-time co- 
director (under California regulations) and a full-time clinical consultant. Dr. Tschirhart has 
over 15 years of experience as a clinical laboratory director, and is a valuable part of the 
laboratory’s new leadership and oversight team. The laboratory’s new leadership team has 
installed an entirely redesigned QMPI Program, as well as robust audit procedures that 
required monthly and quarterly audits. 


D6086 Finding #6 — March 18 Letter Statement 4: “Jn its submission the laboratory states 
“training has occurred,” however, training documentation provided in Ex. A Tab 32 did not 
include all testing personnel listed on the CMS-209 (Ex. L, Tab I ).” 


Theranos Response to Statement 4 — D6086 Finding #6: The training records for this SOP 
provided with the submission included all testing personnel listed on the CMS-209 who run 
tests in the laboratory. Please note that Chi Nguyen is no longer a member of the CLIA 
laboratory (we will update our Form 209 accordingly), which is why she does not appear on 
any of the training documentation. The enclosed training records reflect that all testing 
personnel listed on the CMS-209 have been trained on the laboratory’s enhanced protocol 
requiring review of all new manufacturer package inserts and vendor notifications. (Ex. BB, 
Tabs 14B-14D). 


D6086 Finding #6 — March 18 Letter Statement 5: ‘70 ensure the deficient practice does not 
recur, the laboratory indicated that quarterly audits will be performed and suggested that the 
audits results would be reviewed within the laboratory’s OMPI Program. However, the 
laboratory did not establish the procedure by which these quarterly audits are to be conducted. 
In its submission, the laboratory indicates that a “tracer audit may [emphasis added] be used,” 
but did not provided a protocol for a “tracer audit,” the means by which a “tracer audit” would 
be documented, and whether the results of a “tracer audit” would be the information reviewed 
by the OMPI Program.” 


Theranos Response to Statement 5 — D6086 Finding #6: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-8C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
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audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 F1. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]Joth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB, Tab 7 § 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tabs 7B-7C). 


D6093 


Finding #1 


The March 18 Letter states: “See our reviews of D5447, D5449, D5461, D5481, and D5779.” 
Accordingly, we refer you to our responses to those reviews. 


Finding #2 


D6093 Finding #2 —- March 18 Letter Statement 1: The March 18 Letter states: “See our 
reviews of D5481.” 


Theranos Response to Statement 1 — D6093 Finding #2: We refer you to our response to 
your review of D5481. 


D6093 Finding #2 — March 18 Letter Statement 2: “Zhe submission references ‘Ex. I, Tabs 2- 
6.’ We located these tabs, but found no documentation in Tabs 2, 5, and 6.” 


Theranos Response to Statement 2 — D6093 Finding #2: There was an inadvertent 
copying error for Exhibit I to the initial submission. The laboratory has enclosed the 
applicable documents with this letter, as well as additional documents related to the 
laboratory’s broader analysis of the PT/INR test over the entire time it was run on this 
Siemens BCS XP instrument. (Ex. GG). 


D6093 Finding #2 — March 18 Letter Statement 3: “/he laboratory's submission did not 
include an explanation or documentation of an investigation regarding the adjustment of OC 
ranges.” 


Theranos Response to Statement 3 — D6093 Finding #2: D6093#2, Paragraph k, states 
that “[o]n approximately 9/16/15, the laboratory adjusted the acceptable range for Citrol 3 to 
match the data” and that the “change was implemented without any investigation as to the 
reason for the shift in control values.” The laboratory stopped running PT/INR tests on 
September 17, 2015. The laboratory’s review of this issue has come to the same ultimate 
conclusion. Under the laboratory’s prior leadership and its old QA Program, the laboratory 


79 
TRADE SECRET/CONFIDENTIAL COMMERCIAL INFORMATION 
EXEMPT FROM DISCLOSURE UNDER FREEDOM OF INFORMATION ACT 


FOIA Confidential Treatment Requested by Theranos THER-0534778 
Fed. R. Crim. P. 6(e) material 


Case:228168602 26642020 0d metGHws, Pikechity/AW2DI, Page 82 of 98 


thera Meo 1701 Page Mill Road P 650.838.9292 theranos.com 


Palo Alto, CA 94304 F 650.838.9165 


changed the control values for PT/INR on September 16, 2015 without documenting a proper 
investigation, and in contrast to what is required by the laboratory’s new procedures. The 
laboratory’s new leadership and new QMPI Program do not ascribe to this type of practice, 
as changes to QC values should not occur unless a full and documented investigation shows 
that the change is warranted. The laboratory stopped PT/INR testing on September 17, 
2015, thus limiting the potential effect that this change to control values could have. The 
laboratory has not performed any PT/INR tests since September 17, 2015. As indicated 
above, the laboratory has broadened its investigation of PT/INR tests to encompass the entire 
period during which this BCS XP instrument was in use, which was October 2014 through 
September 2015. Based on that investigation, the laboratory has voided all PT/INR results 
reported during that period. (Ex. AA, Tab 7). Many corrected reports have been transmitted, 
and the remainder are being transmitted. (Ex. KK). Transmission will be complete by March 
31, 2016. The remainder of the transmittals and confirmations of receipt will be provided to 
CMS under separate cover. 


D6093 Finding #2 — March 18 Letter Statement 4: “70 ensure the deficient practice does not 
recur, the laboratory indicated that quarterly audits will be performed and suggested that the 
audits results would be reviewed within the laboratory’s OMPI Program. However, the 
laboratory did not establish the procedure by which these quarterly audits are to be conducted. 
In its submission, the laboratory indicates that a “tracer audit may [emphasis added] be used,” 
but did not provided a protocol for a “tracer audit,” the means by which a “tracer audit” would 
be documented, and whether the results of a “tracer audit” would be the information reviewed 
by the OMPI Program.” 


Theranos Response to Statement 4 — D6093 Finding #2: The laboratory has revised its 
audit procedures to establish the procedures by which audits are to be conducted. (Ex. BB, 
Tab 8). Training on these procedures has occurred. (Ex. BB, Tabs 8B-8C). The revised 
procedures establish that tracer audits must occur monthly; provide the protocol for a tracer 
audit; and require that tracer audits be documented using the tracer audit form, CL FRM 
00046 F2. (Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a 
quarterly audit to cover the quality management systems; provide the protocol for quarterly 
audits; and require that quarterly audits are documented using the audit record form, CL QOP 
00046 Fl. (Ex. BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear 
that “[b]oth tracer and quarterly audits are reviewed, analyzed and presented based on 
findings and analysis” during the QMPI Program meetings. (Ex. BB, Tab 7 § 7.5). Training 
on the revised QMPI procedures has occurred. (Ex. BB, Tabs 7B-7C). 


D6094 
Finding #1 
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The March 18 Letter states: “See our reviews of D5391, D5393, D5791, and D5793.” 
Accordingly, we refer you to our responses to those reviews. 


Finding #2 


The March 18 Letter states: “See our reviews of D5413, D5481, D5801, D5805, and 5821.” 
Accordingly, we refer you to our responses to those reviews. 


D6098 


The March 18 Letter states: “See our review of D5805.” Accordingly, we refer you to our 
response to your review of D5805. 


D6102 


D6102 — March 18 Letter Statement 1: “Zhe laboratory submitted protocols relating to testing 
personnel training (Ex. A, Tab 16, §§ 4.5.3 and 6.4) which state the following: 


- § 4.5.3 ‘Testing personnel/trainee are responsible to ensure that training is properly 
documented using CL-FRM-03016-F 4, Training Attendance Form, approved by the 
designated trainer and stored in the relevant binder.’ 

- $6.4‘... Itis the responsibility of the trainee and trainer to ensure the SOP is accurate.’ 


However, the regulations require that the laboratory director, not the testing personnel, is 
responsible for ensuring that prior to testing patient specimens, all personnel have the 
appropriate training and have demonstrated that they can perform all testing operations to 
provide and report accurate results.” 


Theranos Response to Statement 1 in D6102: The laboratory has revised its training and 
competency procedures to clarify that the laboratory director has the responsibility to ensure 
adherence to the Quality Management System, including the training and competency 
procedures. (Ex. BB, Tab 9 §§ 4.1, 4.7.6, 6.9.9.1). Training on these procedures has 
occurred. (Ex. BB, Tabs 9B-9C). 


D6102 — March 18 Letter Statement 2: “/raining documentation submitted in kx. A, lab 11 
only includes training on protocols related to testing personnel qualification requirements and 
delegation of responsibilities, as well as job descriptions. The laboratory provided no 
documentation or plan to train or retrain testing personel on the Theranos Proprietary 
Device.” 


Theranos Response to Statement 2 in D6102: There appears to have been a 
miscommunication because the TPS 3.5 was fully retired before the survey began on 
September 22, 2015. Because the TPS 3.5 is not being used, and will not be used in the 
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future, the laboratory is not required to train or re-train personnel on tests that used to be run 
on that retired instrument. 


D6102 — March 18 Letter Statement 3: “7raining documents submitted in Ex. A, Tab 17 only 
addressed review of the document control and training and competency protocols, but did not 
include training or retraining documentation for personnel performing testing in the 
venipuncture lab nor did it include an investigation if patient were affected or potentially 
affected when untrained or partially trained testing personnel were performing patent testing.” 


Theranos Response to Statement 3 in D6102: The laboratory previously submitted 
training and competency documentation for all testing personnel as part of its response to 
D6124. The laboratory has updated that documentation. A list showing which testing 
personnel operate each instrument currently in use is enclosed. (Ex. HH, Tab 9A).'* Current 
training and competency documentation for those testing personnel and those instruments is 
also enclosed. (Ex. HH, Tab 9B). 


D6102, paragraph a, concerns the personnel training documentation for the TPS 3.5. The 
TPS 3.5 was fully retired in early-August 2015. The laboratory values CMS’s feedback, 
and takes it very seriously. The laboratory has conducted a thorough re-review of QC data 
for each assay run on the TPS 3.5 from January 1, 2014 until the TPS 3.5 was fully retired in 
early-August 2015. As explained further in the attached patient assessment analysis, the 
laboratory is not satisfied with its old quality assessment program’s ability to effectively flag 
and promptly address QC imprecision, QC failures, and QC trends on the TPS 3.5. (Ex. AA, 
Tab 3). The laboratory has new leadership and a new QMPI Program with robust quality 
monitoring and program improvement procedures. Based on its re-review of QC data for the 
TPS 3.5, the laboratory has determined that its prior QA program failed to satisfactorily 
address these types of QC issues for assays run on the TPS 3.5 in 2014 and 2015. As 
corrective action, the laboratory has, out of an extreme abundance of caution, voided results 
reported for assays run on the TPS 3.5 in 2014 and 2015. (Ex. AA, Tab 3). Many corrected 
reports have been transmitted, and the remainder are being transmitted. (Ex. 

KK). Transmission will be complete by March 31, 2016. The remainder of the transmittals 
and confirmations of receipt will be provided to CMS under separate cover. 


'4 Please note that Chi Nguyen, who is listed on the Form 209, is no longer a member of the 
CLIA laboratory (we will update our Form 209 accordingly), which is why she does not appear 
on any of the training documentation. 
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As set forth in the attached analysis, the laboratory has enclosed training documentation for 
Testing Person #6 (TP6); Testing Person #11 (TP11); and Testing Person #31 (TP31). (Ex. 
HH, Tabs 14, 15, 16). The analysis of these training records is set forth in Ex. AA, Tab 12. 


D6102 — March 18 Letter Statement 4: “Zo ensure the deficient practice does not recur, the 
laboratory indicated that quarterly audits will be performed and suggested that the audits results 
would be reviewed within the laboratory’s OMPI Program. However, the laboratory did not 
establish the procedure by which these quarterly audits are to be conducted. In its submission, 
the laboratory indicates that a “tracer audit may [emphasis added] be used,” but did not 
provided a protocol for a “tracer audit,” the means by which a “tracer audit” would be 
documented, and whether the results of a “tracer audit” would be the information reviewed by 
the OMPI Program.” 


Theranos Response to Statement 4— D6102: The laboratory has revised its audit 
procedures to establish the procedures by which audits are to be conducted. (Ex. BB, Tab 8). 
Training on these procedures has occurred. (Ex. BB, Tabs 8B-8C). The revised procedures 
establish that tracer audits must occur monthly; provide the protocol for a tracer audit; and 
require that tracer audits be documented using the tracer audit form, CL FRM 00046 F2. 
(Ex. BB, Tab 8 § 8.1). In addition, the revised audit procedures require a quarterly audit to 
cover the quality management systems; provide the protocol for quarterly audits; and require 
that quarterly audits are documented using the audit record form, CL QOP 00046 F1. (Ex. 
BB, Tab 8 § 8.2). The QMPI Program QOP has been revised to make clear that “[b]oth 
tracer and quarterly audits are reviewed, analyzed and presented based on findings and 
analysis” during the QMPI Program meetings. (Ex. BB, Tab 7 § 7.5). Training on the 
revised QMPI procedures has occurred. (Ex. BB, Tabs 7B-7C). 


D6108 


The March 18 Letter states: “See our review of D6/15.” Accordingly, we refer you to our 
response to your review of D6115. 


D6111 
The March 18 Letter states: “Based on the laboratory’s submission, this requirement is met.” 
D6115 


D6115 — March 18 Letter Statement 1: “Zhe laboratory submitted a protocol for a Master 
Validation Plan at Ex. FE, Tab 30B which was a different protocol than presented to the surveyor. 
In Ex. E, Tab 1, under the Accuracy section, the submission indicates that ‘some of the 
immunoassays on the TSP [Theranos Proprietary System] were ‘calibrated’ or ‘corrected’ to 
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remove systemic bias between the two methods.’ It is not clear what specific biases were used for 
each analyte which used a corrected results or if the corrected or uncorrected results was 
reported. 


Theranos Response to Statement 1 — D6115: This deficiency concerns the TPS 3.5, which 
was one of the Theranos proprietary technologies in use in 2014 and 2015. The laboratory 
began phasing out the TPS 3.5 in December 2014, and it was fully retired in early-August 
2015. 


The laboratory values CMS’s feedback, and takes it very seriously. The laboratory has 
conducted a thorough re-review of QC data for each assay run on the TPS 3.5 from January 
1, 2014 until the TPS 3.5 was fully retired in early-August 2015. As explained further in the 
attached patient assessment analysis, the laboratory is not satisfied with its old quality 
assessment program’s ability to effectively flag and promptly address QC imprecision, QC 
failures, and QC trends on the TPS 3.5. (Ex. AA, Tab 3). The laboratory has new leadership 
and a new QMPI Program with robust quality monitoring and program improvement 
procedures. Based on its re-review of QC data for the TPS 3.5, the laboratory has 
determined that its prior QA program failed to satisfactorily address these types of QC issues 
for assays run on the TPS 3.5 in 2014 and 2015. As corrective action, the laboratory has, out 
of an extreme abundance of caution, voided results reported for assays run on the TPS 3.5 in 
2014 and 2015. (Ex. AA, Tab 3). Many corrected reports have been transmitted, and the 
remainder are being transmitted. (Ex. KK). Transmission will be complete by March 31, 
2016. The remainder of the transmittals and confirmations of receipt will be provided to 
CMS under separate cover. 


D6115 — March 18 Letter Statement 2: /n addition, the Accuracy section described that the 
“samples spanned the medical decision levels (MDL)”. We are unclear as to the laboratory’s 
definition of “medical decision limits.” No definition of this phrase was found in Section 3 - 
Definitions of the “Method Validation” protocol or in Ex. E:. We also note that the submitted 
protocol does not include a reference to “medical decision limits” as a criterion for determining 
accuracy. 


Theranos Response to Statement 2 — D6115: D6115 concerns the TPS 3.5. The TPS 3.5 
was fully retired before in early-August 2015. References to method validation data in 
documents enclosed with the submission are to validation reports prepared before the survey 
began and before the laboratory’s revised procedures were made effective (those reports were 
either reviewed by CMS or available to CMS during the survey), which were referenced for 
context. In addition, the laboratory has further revised its method verification procedure, 
which now has no reference to “critical medical decision limits.” (Ex. BB, Tab 5). 
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The laboratory’s revised method verification procedure references a “medical decision 
level,” which is defined in the procedure as “[t]he concentration of analyte in blood or bodily 
fluid sample being analyzed at which some medical action is indicated for proper patient 
care. There may be several medical decision limits for a given analyte.” (Ex. BB, Tab 5 § 
3.9). In addition, this revised procedure references CLSI EP28-A3 (Ex. BB, Tab 5 § 10.6), 
which states: “for some analytes, reference intervals have been replaced by decision limits 
[emphasis in original], established by national (or international) consensus. As examples, 
consider cholesterol and glycated hemoglobin.” Training on the laboratory’s revised method 
verification procedure has occurred. (Ex. BB, Tabs 5B-5C). 


D6115 — March 18 Letter Statement 3: “We also note that the laboratory’s submission 
provided no indication why the validation report for SHBG had an effective date of 7/14/14, 
but was not approved by the laboratory director until 9/19/15, even though patient testing 
began 7/28/14.” 


Theranos Response to Statement 3 — D6115: This issue is similar to the issue addressed in 
the discussion above regarding D5407. As reflected by the Lab Director deficiencies 
identified by CMS, the prior lab directors did not provide sufficient oversight over the 
documents cited in D5407 and D6115. This appears to be one of the reasons why these 
documents were not timely reviewed and signed by the prior lab directors. In addition, the 
laboratory has found that its quality assurance oversight over these documents, including 
document control oversight, was not sufficient. The laboratory has hired a new lab director, 
who has approved enhanced document control policies and procedures to address this issue. 
(Ex. BB, Tab 1 §§ 5.1, 5.2, 5.3, 6.1, 6.2, 6.3). Training on those procedures has occurred. 
(Ex. BB, Tabs 1B-1C). The laboratory has also hired a Document Control Manager whose 
responsibilities include tracking new and revised policies and procedures to ensure that they 
are reviewed and approved by the lab director before they become effective. (Ex. HH, Tab 
7C). In addition, the laboratory’s revised QMPI Program procedures and agenda require 
review of procedures that are new, revised, or in process at monthly QMPI Program 
meetings. (Ex. BB, Tab 7 §§ 7.1, 7.5.6). 


D6115 — March 18 Letter Statement 4: The /aboratory’s submission in Ex. E, Tab 1 states: 


The procedure and acceptance criteria outlined in the “Guidance for Industry: 
Bioanalytical Method Validation, 2001” document from IDA was followed to establish 
the analytical measurement range (AMR) (reportable range) for these immunoassays run 
on the TPS. Theranos did not follow the validation plan for immunoassays (ELISAS) as 
outlined in Master Validation Plan for ELISA Assays on Theranos Devices. 
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The laboratory’s response does not include a copy of the FDA protocol, and does not state in the 
submitted master validation protocol, CL PLN-14002, Rev. A, that the I'DA protocol should be 
used to determine AMR. 


The laboratory submitted a protocol for a Master Validation Plan at Ex. FE, Tab 30B which 
requires the %CV of replicates to be not more than 20% (25% at the lower and upper limits of 
detection). The submitted protocol was different than the protocol presented to the surveyor at 
the time of the onsite survey which required the %CV of replicates to be not more than 15% 
(20% at the lower and upper limits of detection). The %CV in the protocol presented to the 
surveyor at the time of the onsite survey matched the %CV of the predicate devices. Based on the 
protocol submitted in the submission, it is unclear as to why the TPS, which is covered by the 
Master Validation Plan, can now have %CV’s for replicates greater than the predicate devices. 
No explanation for the change in ¥%CV was submitted. In addition, we noted that the information 
submitted in Ex. , Tab 1 did not include a 20-day precision study as required by the submitted 
protocol. 


The laboratory's submitted protocol requires that 120 samples be used to determine Reference 
Intervals (see Ex. Ek, Tab 30B), Ex. FE, Tab 1 states: 


However, the validation plan noted that the reference range would be established with 
120 samples per partition. However, the study followed CLSI procedure for transferring 
a reference range - the process by which one adapts a previously established reference 
range to a new analytical method. Furthermore, the transferred reference range was 
verified following CLSI guidelines - namely “the process by which one ensures, with 
reasonable confidence, using a relative small number of reference individuals (eg, 

n=20), that a reference interval established elsewhere, or transferred from another study, 
can be used locally. 


We question the laboratory's procedure for “transferring a reference range” since the 
laboratory provided no reference document to support “transferring a reference range.” We 
note that the submitted protocol does not state in Section 18, Reference Range, that the 
laboratory can “transfer” a reference range. It is unclear as to how the laboratory determined 
that the predicate method's reference range could be adapted or “transferred” to the Theranos 
Proprietary Devices, which used different methodology than the predicate method. 


Theranos Response to Statement 4 — D6115: D6115 concerns the TPS 3.5. The TPS 3.5 
was fully retired in early-August 2015. In addition, the laboratory stopped using its other 
LDTs, and does not have plans to resume using them as it transitions to compliance with 
FDA quality systems requirements for its LDTs and introduces next generation technologies. 
Accordingly, neither the master validation plan for ELISA assays nor “Guidance for 
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Industry: Bioanalytical Method Validation, 2001” are being used by the laboratory. The 
laboratory does not plan to use either of those documents, plans or procedures in the future. 


The laboratory will ensure that all future validation plans are documented and supported in 
accordance with the laboratory’s new quality systems and procedures. Significantly, the 
laboratory has brought in an entirely new leadership team that is directly involved with 
implementing and monitoring the laboratory’s new procedures. The new laboratory director, 
Dr. Kingshuk Das, began working for the laboratory full-time as of March 14, 2016. Dr. Das 
is also one of the laboratory’s technical supervisors, which further reflects his level of 
involvement in the laboratory. In addition, the laboratory has hired Donald Tschirhart, M.D 
to be a full-time co-director (under California regulations) and a full-time clinical consultant. 
Dr. Tschirhart has over 15 years of experience as a clinical laboratory director, and is a 
valuable part of the laboratory’s new leadership and oversight team. The laboratory’s new 
leadership team has installed an entirely redesigned QMPI Program, as well as robust audit 
procedures that required monthly and quarterly audits. 


The laboratory values CMS’s feedback, and takes it very seriously. The laboratory has 
conducted a thorough re-review of QC data for each assay run on the TPS 3.5 from January 
1, 2014 until the TPS 3.5 was fully retired in early-August 2015. As explained further in the 
attached patient assessment analysis, the laboratory is not satisfied with its old quality 
assessment program’s ability to effectively flag and promptly address QC imprecision, QC 
failures, and QC trends on the TPS 3.5. (Ex. AA, Tab 3). The laboratory has new leadership 
and a new QMPI Program with robust quality monitoring and program improvement 
procedures. Based on its re-review of QC data for the TPS 3.5, the laboratory has 
determined that its prior QA program failed to satisfactorily address these types of QC issues 
for assays run on the TPS 3.5 in 2014 and 2015. As corrective action, the laboratory has, out 
of an extreme abundance of caution, voided results reported for assays run on the TPS 3.5 in 
2014 and 2015. (Ex. AA, Tab 3). Many corrected reports have been transmitted, and the 
remainder are being transmitted. (Ex. KK). Transmission will be complete by March 31, 
2016. The remainder of the transmittals and confirmations of receipt will be provided to 
CMS under separate cover.» 


'S In discussing establishment of precision for an LDT, the March 18 letter references a 
comparison of %CVs between an LDT and “predicate devices.” We note, however, that CLIA 
regulations do not contain any regulation requiring that there be any correlation or equivalence of 
%CV between LDTs and predicate devices. Rather, the regulations require only that the 
precision for the test must be established for the LDT itself. 42 C_F.R. § 493.1253(b)(2). It also 
is generally accepted that accuracy is the only required performance specification that typically 
requires a correlation study. 42 C.F.R. § 493.1253(b)(2). 
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D6115— March 18 Letter Statement 5: “Zhe laboratory’s submission does not indicate that the 
% Recovery and Allowable Bias citations had been addressed..... Because the laboratory has not 
shown that it can follow its own validation protocols, it is unclear whether the laboratory's quality 
assessment mechanisms could monitor the laboratory’s corrective actions and ensure this 
deficient practice does not recur.” 


Theranos Response to Statement 5— D6115: D6115 concerns the TPS 3.5. The TPS 3.5 
was fully retired in early-August 2015. In addition, neither the master validation plan for 
ELISA assays nor the master validation plan for chemistry assays are being used by the 
laboratory. The laboratory does not plan to use either of those documents, plans or 
procedures in connection with clinical testing in the future. 


The laboratory values CMS’s feedback, and takes it very seriously. The laboratory has 
conducted a thorough re-review of QC data for each assay run on the TPS 3.5 from January 
1, 2014 until the TPS 3.5 was fully retired in early-August 2015. As explained further in the 
attached patient assessment analysis, the laboratory is not satisfied with its old quality 
assessment program’s ability to effectively flag and promptly address QC imprecision, QC 
failures, and QC trends on the TPS 3.5. (Ex. AA, Tab 3). The laboratory has new leadership 
and a new QMPI Program with robust quality monitoring and program improvement 
procedures. Based on its re-review of QC data for the TPS 3.5, the laboratory has 
determined that its prior QA program failed to satisfactorily address these types of QC issues 
for assays run on the TPS 3.5 in 2014 and 2015. As corrective action, the laboratory has, out 
of an extreme abundance of caution, voided results reported for assays run on the TPS 3.5 in 
2014 and 2015. (Ex. AA, Tab 3). Many corrected reports have been transmitted, and the 
remainder are being transmitted. (Ex. KK). Transmission will be complete by March 31, 
2016. The remainder of the transmittals and confirmations of receipt will be provided to 
CMS under separate cover. 


The laboratory is capable of ensuring that the deficient practice in D6115 does not recur. 
Significantly, the laboratory has brought in an entirely new leadership team that is directly 
involved with implementing and monitoring the laboratory’s new procedures. The new 
laboratory director, Dr. Kingshuk Das, began working for the laboratory full-time as of 
March 14, 2016. Dr. Das is also one of the laboratory’s technical supervisors, which further 
reflects his level of involvement in the laboratory. In addition, the laboratory has hired 
Donald Tschirhart, M.D to be a full-time co-director (under California regulations) and a 
full-time clinical consultant. Dr. Tschirhart has over 15 years of experience as a clinical 
laboratory director, and is a valuable part of the laboratory’s new leadership and oversight 
team. The laboratory’s new leadership team has installed an entirely redesigned QMPI 
Program, as well as robust audit procedures that required monthly and quarterly audits. For 
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example, § 8.1.1.3 of the method verification procedure lays out the requirements for a 
recovery design as part of an IVD verification. (Ex. BB, Tab 5 § 8.1.1.3). 


D6115 — March 18 Letter Statement 6: “/x. contains a list of patient specimen accession 
numbers for which the laboratory ‘identified reports’, but did not specify, or submit 
documentation to show that corrected reports were generated and issued.” 


Theranos Response to Statement 6 — D6115: As described above, the laboratory has 
revised its patient assessment analysis based upon a thorough re-review of QC data for the 
TPS 3.5. Based on that analysis, the laboratory has, out of an extreme abundance of caution, 
voided all results reported for assays run on the TPS 3.5 in 2014 and 2015. (Ex. AA, Tab 3). 
Many corrected reports have been transmitted, and the remainder are being transmitted. (Ex. 
KK). Transmission will be complete by March 31, 2016. The remainder of the transmittals 
and confirmations of receipt will be provided to CMS under separate cover. 


D6124 


D6124 — March 18 Letter Statement 1: “/’x. A, Tab 16, § 7.3 of the submission contains a 
competency assessment protocol, Annual Competency. However, the protocol does not address 
the cited deficiency that direct observation was required for instrument maintenance and 
function checks. The protocol at § 1.1.7 (page 12) allows for a choice of validation method of 
which direct observation is one choice. The submitted laboratory protocol also requires the use 
of CL FRM 03016-F3 for annual competency. This form also allows the assessor to choose the 
validation method for competency assessment. Neither the submitted protocol nor the form to 
document competency assessment indicate that the six required procedures for CLIA competency 
assessment are required.” 


Theranos Response to Statement 1 in D6124: The laboratory has revised its training and 
competency procedures to clarify that the performance of maintenance and function checks 
must be directly observed (Ex. BB, Tab 9 §§ 6.9.4.4, 9.1), and to expressly require each of 
the procedures required by 42 CFR § 493.1451(b)(8)(iv). (Ex. BB, Tab 9 §§ 6.9.4). Training 
on these procedures has occurred. (Ex. BB, Tabs 9B-9C). 


D6124 — March 18 Letter Statement 2: “/x. L, Zab 4 of the submission states: ‘All testing 
personnel currently performing tests have completed competency testing with direct observation 
of, among other things, maintenance and function checks for the tests they are performing.’ We 
are unable to determine if the documentation is complete as there is no information submitted 
about which testing personnel were performing testing on which instrument(s). Because the 
laboratory has not shown that it has corrected this deficient practice, we are also unable to 
determine whether the laboratory’s quality assessment mechanisms can monitor the laboratory’s 
corrective actions and ensure this deficient practice does not recur.” 
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Theranos Response to Statement 2 in D6124: The laboratory’s submission included 
training and competency documentation for all testing personnel on the instruments they 
were operating at that time. The laboratory subsequently made some minor staffing changes. 
A list showing which testing personnel operate each instrument currently in use is enclosed. 
(Ex. HH, Tab 9A). Current training and competency documentation for those testing 
personnel and those instruments is also enclosed. (Ex. HH, Tab 9B). That documentation 
shows that all testing personnel have completed competency testing for the tests they are 
performing. (Ex. HH, Tabs 9A-9B). 


The laboratory is capable of ensuring that the deficient practice in D6124 does not recur. 
Significantly, the laboratory has brought in an entirely new leadership team that is directly 
involved with implementing and monitoring the laboratory’s new procedures. The new 
laboratory director, Dr. Kingshuk Das, began working for the laboratory full-time as of 
March 14, 2016. Dr. Das is also one of the laboratory’s technical supervisors, which further 
reflects his level of involvement in the laboratory. In addition, the laboratory has hired 
Donald Tschirhart, M.D to be a full-time co-director (under California regulations) and a 
full-time clinical consultant. Dr. Tschirhart has over 15 years of experience as a clinical 
laboratory director, and is a valuable part of the laboratory’s new leadership and oversight 
team. The laboratory’s new leadership team has installed an entirely redesigned QMPI 
Program, as well as robust audit procedures that required monthly and quarterly audits. 


D6170 


D6170 — March 18 Letter Statement 1: “/x. J, Jab 46 contains lists of patient specimen 
accession numbers for which the laboratory intended to issue corrected test reports. The 
laboratory provided no documentation to indicate corrected reports were generated and issued.” 


Theranos Response to Statement 1 in D6170: Many corrected reports have been 
transmitted, and the remainder are being transmitted. (Ex. KK). Transmission will be 
complete by March 31, 2016. The remainder of the transmittals and confirmations of 
receipt will be provided to CMS under separate cover. 


D6170 — March 18 Letter Statement 2: “Zhe submission further states: 


- “Based on comprehensive review, multiple examples of failed OC [quality control] without 
the appropriate documentation with no investigation or corrective action was identified 
which is now addressed with revised SOPS and training.” 

- “... the review identified multiple examples where technical service provided PM and that 
service was not documented in addition to failed QC run as part of the service which is 
now addressed with revised SOPs and training.” 
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The laboratory provided no documentation of investigation or corrective action for these OC 
failures and lack of maintenance documentation.” 


Theranos Response to Statement 2 in D6170: There appears to have been a 
miscommunication here because the laboratory stopped running tests on the Fortessa and 
Canto before the survey began on September 22, 2015, and has not run any tests on the 
instruments since then. Because the Fortessa and Canto instruments are not in use, and have 
not been in use since the survey began, there have not been any new QC failures or 
maintenance issues that required investigation or corrective action under the laboratory’s new 
procedures. The “review” discussed in the submission was simply intended to refer to the 
analysis conducted to address the deficiency identified by CMS. The issue addressed by the 
“review” — i.e., failing to document corrective action for QC fails for the Fortessa and Canto 
— is the same issue identified by CMS in its deficiency, and thus did not constitute a new 
issue Or Occurrence. 


D6170 — March 18 Letter Statement 3: “x. A, Zab 10, § 5.10 states: ‘Unlicensed laboratory 
personnel are responsible for performing the activities listed below, under direct and constant 
supervision by licensed test personnel, with strict adherence to regulatory requirements ...’ 
California. The laboratory relied on its written policies and procedures to ensure that 
appropriately licensed testing personel were performing and reporting patient test results. 
However, if the laboratory’s policies and procedures are inconsistent with state requirements, 
we question whether this mechanism would prevent the deficient practice from recurring, and 
question the laboratory's document control system to ensure accurately written policies and 
procedures.” 


Theranos Response to Statement 3 in D6170: The laboratory’s policy does not permit 
unlicensed personnel to run patient tests, and the laboratory did not intend to imply 
otherwise. To eliminate any confusion or potential for confusion, however, the laboratory has 
revised this procedure to remove the language identified by CMS. (Ex. BB, Tab 6 § 5.10). 


D6171 


D6170 — March 18 Letter Statement 1: “/x. 1, Zab 3 of the submission states: ‘TP 14 has been 
retrained to ensure that TP 14 only performs activities within the scope of TP 14’s job description 
as a clinical laboratory associate, under the supervision of testing personnel. Because TP 14 is 
not testing personnel, the high complexity personnel educational requirements do not apply.’ The 
CLIA job responsibilities in the job description provided at Ex. L, Tab 3 does include CLIA 
regulatory responsibilities for testing personnel, and as such TP14 was required to meet the 
educational requirements. The laboratory’s submission does not provide any additional 


91 
TRADE SECRET/CONFIDENTIAL COMMERCIAL INFORMATION 
EXEMPT FROM DISCLOSURE UNDER FREEDOM OF INFORMATION ACT 


FOIA Confidential Treatment Requested by Theranos THER-0534790 
Fed. R. Crim. P. 6(e) material 


Case: 228168022664E)B0 Bodo meaiGeee, Ditechtty/AW2, Page 93 of 94 


thera Meo 1701 Page Mill Road P 650.838.9292 theranos.com 


Palo Alto, CA 94304 F 650.838.9165 


documentation that establishes that TP 14 was qualified to perform high complexity testing. TP 14 
remains unqualified to perform high complexity testing. 


Theranos Response to Statement 1 in D6170: The job description for TP14 has been 
revised to be consistent with the job description of a Clinical Laboratory Associate in the 
laboratory’s revised personnel procedures. (Ex. HH, Tab 7B). The laboratory has confirmed 
that TP14 has not performed any test analyses between November 2015 and his execution of 
this revised job description. (Ex. HH, Tab 7A). 


D6178 


D6178 — March 18 Letter Statement 1: “x. F contains lists of patient specimen accession 
numbers for which the laboratory intended to issue corrected test reports. The laboratory provided 
no documentation to indicate corrected reports were generated and issued.” 


Theranos Response to Statement 1 in D6178: Many corrected reports have been 
transmitted, and the remainder are being transmitted. (Ex. KK). Transmission will be 
complete by March 31, 2016. The remainder of the transmittals and confirmations of 
receipt will be provided to CMS under separate cover. 


D6178 — March 18 Letter Statement 2: ‘Zo ensure the deficient practice does not recur, the 
laboratory indicated that quarterly audits will be performed and suggested that the audits results 
would be reviewed within the laboratory’s OMPI Program. However, the laboratory did not 
establish the procedure by which these quarterly audits are to be conducted. In its submission, 
the laboratory indicates that a “tracer audit may [emphasis added] be used,” but did not 
provided a protocol for a “tracer audit,” the means by which a “tracer audit” would be 
documented, and whether the results of a “tracer audit” would be the information reviewed by 
the OMPI Program.” 


Theranos Response to Statement 2 — D6178: The laboratory has revised its audit procedures 
to establish the procedures by which audits are to be conducted. (Ex. BB, Tab 8). Training on 
these procedures has occurred. (Ex. BB, Tabs 8B-8C). The revised procedures establish that 
tracer audits must occur monthly; provide the protocol for a tracer audit; and require that tracer 
audits be documented using the tracer audit form, CL FRM 00046 F2. (Ex. BB, Tab 8 § 8.1). 
In addition, the revised audit procedures require a quarterly audit to cover the quality 
management systems; provide the protocol for quarterly audits; and require that quarterly 
audits are documented using the audit record form, CL QOP 00046 F1. (Ex. BB, Tab 8 § 8.2). 
The QMPI Program QOP has been revised to make clear that “[b oth tracer and quarterly audits 
are reviewed, analyzed and presented based on findings and analysis” during the QMPI 
Program meetings. (Ex. BB, Tab 7 § 7.5). Training on the revised QMPI procedures has 
occurred. (Ex. BB, Tabs 7B-7C). 
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Sincerely, 


Dr. Kingshuk Das 


Laboratory Director 
Theranos, Newark California Laboratory 
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